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A Code Restructuring Tool to help Scaffold 
Novice Programmers 

Stuart Garner 

Edith Cowan University 

Pearson St., Churchlands 6018, Western Australia 
E-mail: s . gamer@ecu. edu.au 



This paper concerns a new software tool called CORT (code restructuring tool) that 
has been developed by the author to help students learn programming. The paper 
begins by discussing the difficulties that students face when learning to program and 
the use of part complete solutions as a teaching and learning method that reduces the 
cognitive load that students experience. 

CORT has been developed to support this use of part complete solutions and its 
features are outlined. When used by a student, a part complete solution to a given 
programming problem is displayed in one window and possible lines of code that 
can be used to complete the solution are displayed within another window. The lines 
can easily be moved between the windows in order to complete the solution and the 
solution then transferred to the target programming environment for testing 
purposes. 

Finally, the use of CORT with both undergraduate and postgraduate students at 
Edith Cowan University is described, preliminary feedback from students indicating 
that CORT is easy to use and that they perceive that it is helping them in their 
learning of programming. Four different methods of using CORT have been 
identified and these will be the subject of future research. 



Keywords: Scaffolding, Programming, Flexible Learning. 



1 Introduction 

Learning to write computer programs is not easy [3, 18] and this is reflected in the low levels of achievement 
experienced by many students in first programming courses. For example, Perkins, Schwartz et al [17] state that: 



Students with a semester or more of instruction often display remarkable naivete about the 
language that they have been studying and often prove unable to manage dismayingly simple 
programming problems. 



and King, Feltham et al [8] state that: 



even after two years of study , many students had only a rudimentary understanding of 
programming 



Over the years since the advent of high level programming languages in the 1960s, much has been written about 
the problems that students have in learning programming and many ideas and initiatives have been put forward 
for improvements in the teaching and learning process with varying degrees of success. In practice, the ways in 
which teaching and learning takes place in the domain of programming have changed little and many students 
still find the learning of programming a very difficult process. The challenge of learning programming in 
introductory courses lies in simultaneously learning: general problem solving skills; algorithm design; program 



design; a programming language in which to implement algorithms as programs; and an environment to support 
the program design and implementation [6]. In addition, students need to learn testing and debugging techniques 
to validate programs and to identify and fix problems that they may have within their programs. 

Additionally, we are moving ever more rapidly to use more student centred and flexible learning methods within 
the teaching and learning process. This means that our instructional design for programming courses needs to 
take notice of these moves and utilise these methods. Fortunately technological improvements have also been 
significant over the last few years enabling us to more easily produce engaging courseware that can help students 
studying in a flexible learning mode. As courseware designers, we can produce electronic scaffolds to help 
students in their learning processes when they are studying on their own with limited access to a human tutor. 



2 Use of Worked Examples in the Teaching and Learning of Problem 
Solving and Programming 



There are several methods used in the teaching and learning of programming and one of these is to utilise 
worked examples. Several researchers have experimented with the use of worked examples in place of 
conventional instruction and found strong advantages. In the domain of algebra, S weller and Cooper [19] 
suggested that students would learn better by studying worked examples until they had "mastered" them rather 
than attempting to solve problems as soon as they had been presented with, or familiarised themselves, with new 
material. In their research, students studied worked examples and teachers answered any questions that the 
students had. Students then had to explain the goal of each problem together with the steps involved in the 
solution and then complete similar problems until they could be solved without errors. Sweller and Cooper found 
that this method was less time-consuming than the conventional practice-based model and that students made 
fewer errors in solving similar problems than students who were exposed to the conventional practice-based 
model of instruction. There was no significant difference between the "worked example" group and the 
"conventional" problem solving group when they attempted to solve novel problems and it was therefore 
concluded that learning was more efficient and yet no less effective when this worked example method was 
used. 

Worked examples are heavily used within the “reading” method of learning programming. According to Van 
Merrienboer et al [22, 23] the reading approach emphasises the reading, comprehension, modification and 
amplification of non-trivial, well-designed working programs. However, they also suggest that presenting 
worked examples to students is not sufficient as the students may not “abstract” the programming plans from 
them, a plan being a stereotyped sequence of computer instructions as shown in figure 1. 

“Mindful” abstraction of plans is required by the voluntary investment of effort and the question then arises as to 
how we can get students to study the worked examples properly. In practice, students tend to rush through the 
examples, even if they have been asked to trace them in a debugger, as they often believe that they are only 
making progress in their learning when they are attempting to solve problems. 

Lieberman [10] suggests that students should annotate worked examples with information about what they do or 
what they illustrate. Another suggestion is to use incomplete, well-structured and understandable program 
examples that require students to generate the missing code or “complete” the examples. This latter approach 
forces students to study the incomplete examples as it would not be possible for their completion without a 
thorough understanding of the examples’ workings. An important aspect is that the incomplete examples are 
carefully designed as they have to contain enough “clues” in the code to guide the students in their completion. It 
is suggested that this method facilitates both automation, students having blueprints available for mapping to 
new problem situations, and schemata acquisition as they are forced to mindfully abstract these from the 
incomplete programs [24]. 

In one study, two groups of 28 and 29 high-school students from grades 10 to 12 participated in a ten lesson 
programming course using a subset of COMAL-80 [24]. One group, the “generation” group, followed a 
conventional approach to the learning of programming that emphasised the design and coding of new programs. 
The other group, the “completion” group, followed an approach that emphasised the modification and extension 
of existing programs. It was found that the completion group was better than the generation group in 
constructing new programs. It was found that the percentage of correctly coded lines was greater and that 
looping structures were more often combined with correct variable initialisation before a loop together with the 
correct use of counters and accumulators within the loop. It would appear that the completion strategy had 
indeed resulted in superior schemata formation for those students within that group. In addition, the completion 
group used superior comments in connection with the scope and goals of the programs, indicating that they had 
developed better high-level templates or schemata. It was noted in the study however that both groups were 
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equal in their ability to interpret programs and that this might indicate that students in the completion group do 
not understand their acquired templates. It is then suggested that future completion strategies should include the 
annotation of the examples by students with details of what they are supposed to do and details of the templates 
(plans) that are being used. 



PROGRAM Example(Input, Output); 

VAR Sum, Count, Num : INTEGER; 



Average : REAL; 

BEGIN 




Figure 1 



A side effect of the research was also noted. The drop-out rate from the completion group was found to be lower 
than for the generation group, particularly for female students with low prior knowledge. It was suggested that 
perhaps the generation of complete programs is perceived as a difficult and menacing task and that the 
completion strategy overcomes this difficulty. 

The stimulation of the “mindful of abstraction” of schemata in students can possibly be improved further 
requiring them to also annotate the solutions with details of the scope and goals of the solutions and to answer 
questions on the inner workings of the solutions. The “degree” of completion of the solutions is an important 
aspect within the completion strategy and in some later work [23] examples are given of completion assignments 
that might be used early and later in a programming course. In an early part of a course, an example may indeed 
be complete and include explanations and a question on its inner workings. In the latter part of*a course, the 
example may be largely incomplete and include a question on its workings and an instructional task. Between 
these two extremes, examples will have varying degree of completeness and in all cases, the incomplete 
examples are acting as scaffolds for the students. 



3 The Cloze Procedure 



A scaffolding tool called CORT (Code Restructuring Tool) has been produced that allows students to fill in lines 
of missing code from programs and this method is based upon the cloze procedure. The term is derived from 
“closure”, a Gestalt psychology term referring to the human tendency to complete a familiar but not quite 
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finished pattern [2]. The use of cloze was first used to measure comprehension in English readability [9] 
however it has also been used in the teaching and learning of programming as a way of measuring student 
understanding of programs [7, 20]. Such program comprehension tests are constructed by replacing some of the 
“words” or tokens by blanks and requiring students to fill in the blanks during a test. The use of the cloze 
procedure in testing was found to correlate well with conventional comprehension, question - answer, type 
quizzes and is also much easier to create and administer, see for example the work of Cook, Bregar et al [2]. 
Other researchers have experimented with the testing of program comprehension by omitting complete lines of 
code from programs and requiring students to fill in those lines [5, 13, 14, 15, 16]. Norcio found that students 
were more likely to supply correct statements if they had been omitted within a logic segment rather than from 
the beginning of a segment. This is consistent with the chunking hypothesis [12] that specifies that the first 
element of a chunk provides the key to the contents of the entire unit. Ehrlich looked at the differences between 
experts and novices in filling in missing lines within various programming plans and, as expected, found that the 
experts filled in the lines correctly taking into account the surrounding plan whereas novices had more difficulty. 
In the various experiments in program comprehension using the cloze procedure, the students had to fill in the 
lines of code without being given a selection of lines to choose from. In some work done in an area unrelated to 
programming, students were expected to create an essay using a file of statements, only some of which were 
relevant to the topic [4]. The students were expected to copy and paste only the statements which they believed 
to be relevant and then to link them with their own text and it was suggested that learners would consolidate their 
understanding of the topics by having to actively evaluate all possible statements. The file of statements was 
acting as a scaffold to student learning. 

Although the literature suggests that the cloze procedure has only been used in measuring program 
comprehension, it appears that it could prove useful as a way of scaffolding student learning of programming. 
An incomplete solution to a programming problem could be given to a student together with a choice of 
statements that might be used in the solution. The student would then have to study the incomplete solution and 
the choice of statements and decide which statements to use and where to put them. CORT uses this idea making 
the mechanics of placing the statements into the incomplete solution very straightforward for the student and 
eliminating typing errors and therefore also syntax errors. 



4 The Code Restructuring Tool (CORT) 



CORT has been designed to support the “completion” method of learning to program and it was decided that the 

following features would be required in the first prototype: 

• Support for part-complete solutions to programming problems. Such solutions help in schemata creation and 
also reduce cognitive load. 

• A mechanism so that missing statements can easily be inserted into a part-complete solution and also moved 
within that solution. This provides scaffolding for students. 

• A facility so that students can add and amend lines of code. This would allow scaffolding to be reduced and 
for students to add more of their own code. 

• For visual programming, a facility for students to easily view the target interface. The interface should be 
annotated with the various object names thereby reducing any split-attention effect and helping reduce 
cognitive load [1]. 

• A facility to access tutor created questions concerning the programming problems being attempted and for 
students to enter answers to those questions. This will promote reflection and higher order thinking. 

• A facility to easily transfer a completed solution from CORT to the target programming environment. 

• A facility to easily transfer programming code from the target programming environment back into CORT 
for further amendment. 



4.1 The CORT Design 

The user interface of CORT has been designed taking into consideration the three issues that have been 
suggested by Marcus [11] as being fundamental to interface design, namely development, usability, and 
acceptance. The interface for CORT is shown in figure 2. 
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The ways in which the CORT design supports the list of required features are described in the following table. 



Feature 


Support in CORT Design 


Support for part-complete solutions to 
programming problems 


The part-complete solutions are automatically 
loaded into the right hand window and possible 
statements into the left hand window. Students 
load these from a file. 


A mechanism so that missing statements can 
easily be inserted into a part-complete solution 
and also moved within that solution 


Two buttons in the middle of the screen will move 
lines between the windows. One line, or several 
lines can be selected and moved across. 


A facility so that students can add and amend 
lines of code 


A simple editor is provided so that students can 
add their own lines or amend existing lines. 


For visual programming, a facility for students to 
easily view the target interface 


Access to this feature is via a button on the fixed 
toolbar. 


A facility to access tutor created questions on the 
workings of the programming examples and to 
enter student answers 


Access to this feature is via a button on the fixed 
toolbar. Student answers are automatically saved. 


A facility to easily transfer a completed solution 
from CORT to the target programming 
environment 


This is provided by a button on the main toolbar. 
A single click will copy the contents of the right 
hand window to the Windows clipboard ready for 
pasting into the Visual BASIC programming 
environment. 





A facility to easily transfer programming code 
from the target programming environment back 
into CORT for further amendment 



This is provided by a button on the main toolbar. 
A single click will paste the contents of the 
Windows clipboard into the right hand window, 
overwriting what is there. 



4.2 Use of CORT by Students 



A student would typically use CORT as follows: 

1 . A student loads in a CORT file and the two windows display a part-complete solution to a problem together 
with possible lines to be used. There is a facility available for the contents of the two windows to be printed 
out. 

2. The student can view the problem statement and the Visual BASIC solution interface by clicking on the 
appropriate buttons on the fixed toolbar. The problem statement may have already been provided to the 
student in the form of a handout, however there is also a facility to print it from within CORT. 

3. The student moves certain lines from the left hand window to the right hand window in an attempt to 
complete the solution. Lines can be moved up or down, and indented or outdented in the right hand window. 
Some problems have too many lines in the left hand window, some of those lines being incorrect. 

4. If necessary, the student can invoke a simple editor to amend, add or delete lines of code. 

5. The student clicks on the appropriate button to copy the contents of the right hand window to the Windows 
clipboard. 

6. The student invokes Visual BASIC and loads the file that contains the interface for the solution. This is in 
effect the Visual BASIC solution to the problem without the lines of code and was created by the tutor. 

7. The student pastes the contents of the Windows Clipboard into the Visual BASIC editor and tests the 
program to determine if it works correctly. Use is made of the trace and debugging facilities of Visual 
BASIC. These facilities provide an insight to the workings of the notional machine. 

8. If the student finds a problem with the working of the program, they can return to CORT and make the 
changes to the code there. 

9. The student repeats steps 3 to 8 until they have a working program. 

10. The student answers the tutor’s questions concerning the programming problem that they have just 
attempted. 

4.3 Initial Student Feedback 

CORT has been used for one semester with both undergraduate and postgraduate students in the Faculty of 

Business and Public Management. The particular units are in the area of software development and the language 

that the students learn is Visual BASIC. 

Each week the students have to undertake completion programming exercises using CORT and after each 

problem they were asked to comment on the use of CORT for the particular problem that they had just finished. 

The data was collected on-line through the Web and below are some of the comments that were received: 

/*■“ 

1 . It's very helpful. I can see the interface of the program before actually running it. 

2. I think CORT is a very useful tool to play around the codes. It saves me time copying and pasting. 

3. Considering the increased workload as the semester progresses it is a bit of a relief that the exercises are 
much easier with the "fill in the gap" type format in CORT. 

4. Without CORT, it’s sure that 111 have a lot trouble with this particular problem, which focuses on arrays (a 
difficult topic). Thanks CORT... 

5. CORT was useful in that the part solution helped to understand the logic of VB code 

6. CORT is useful . However, I have used the unit text to try to understand the indentation format when writing 
the code. The directional keys are great for editing the code to meet the required format. 
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7. This was a challenge! I think that CORT is useful so long as I am not tempted to simply manipulate code 
until the program runs. If I were having to write programs from scratch I would use CORT so as to format 
and manipulate code and modules or sub procedures etc. 

5 Conclusions 

As can be seen from the above, the initial feedback on the use of CORT has been favourable. We have found that 
students can undertake two or three small programming problems within a one hour tutorial whereas without 
CORT they could only undertake one such problem. Also, without using CORT students often never manage to 
successfully complete their assigned problems and this certainly affected their motivation. 

By using CORT, students do not have to be concerned with the design of programming interfaces which 
considerably reduces the cognitive load in the initial stages of learning programming. Also, the reduction of 
“split attention affect” by labelling all the objects with their names has been very popular with the students. 

The above has described a preliminary study of the use of CORT and it has been undertaken to determine its 
suitability and to fine tune some of its features. CORT can be used in several ways and four distinct methods 
have now been identified. These will be the subject of further research. The four methods are as follows: 

1. AH of the lines that are required to complete a program are made available in the left hand window of 
CORT. There are no extra lines displayed in the left hand window. 

2. AH of the lines that are required to complete a program are made available in the left hand window of 
CORT. There are also additional lines displayed in the left hand window that are not required within the 
program. The extra lines are similar to the required lines, however they are incorrect and act as “red 
herrings”. 

3. Some of the lines that are required to complete a program are made available in the left hand window of 
CORT. Other lines that are required for the program completion need to be keyed in by the student. 

4. None of the lines that are required to complete a program are made available in the left hand window of 
CORT. All of the lines that are required for the program completion need to be keyed in by the student. 
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Information technology integrated instruction is the education tendency in the 
future, and it is also an important issue in the development of education in 
Taiwan. An assessment framework is needed during experimenting with 
integrated instruction and when integrated instruction is officially implemented. 

The framework can help us understand the implementation and provide 
information for future reference. This article proposes an framework for 
assessing information technology integrated instruction. The framework 
includes kernel and periphery parts. Kernel part refers to the whole teaching 
process, including information technology, curricula, learning materials, 
instructional strategies, learning activities, and evaluation. Periphery part refers 
to the surroundings situation, including teachers, students, information 
specialists, administrators, classroom settings, computer laboratories, campus 
instruction network, Internet, digital materials, and instruction/leaming software. 
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1 Introduction 

The rapid development of information technology (IT) has not only brought about major effect on economy 
and industry but also made a great impact on society and education. In particular, the prevalent use of 
computers and the rapid development of the Internet have gradually changed our life style and pattern, with 
their impact on education being unprecedented. Many educators and policy makers believe that technology 
can be a catalyst for educational reform [3,4,10]. They suggest that the use of technology in classrooms will 
shift the roles of teachers and students. Teachers will act more like facilitators by helping students access 
information, process it, and communicate their understanding [4]. 

Beginning the 2001 academic year, Taiwan will implement phase-by -phase the nine-year integrated 
curriculum for its elementary and junior high schools [11]. To cultivate students’ basic ability to “apply 
technology and information”, the new curriculum will have to emphasize integrating IT into the teaching of 
various courses. Amid this major reform of curriculum, the Computer Center of the Ministry of Education 
has planned for the integration of information curriculum with other areas of learning [7]. At the same time, 
it has selected 18 elementary and junior high schools in which teaching experimentations will be carried out 
[1]. Therefore, an assessment framework is needed during experimenting with integrated instruction and 
when integrated instruction is officially implemented. The framework can help us understand the 
implementation and provide information for future reference. 



2 The essence of information technology integrated instruction 

The United States has implemented IT integrated instruction for years. Many educators are now actively 
using technology along with effective teaching strategies to integrate technology into their curriculum [9]. In 
contrast, IT integrated instruction is still a newly heard noun in Taiwan. Many teachers are unfamiliar with it, 
and some think of it as another name for computer-assisted instruction (CA1). Information technology has 



developed rapidly, and the role of IT in education has changed over these years, from being an auxiliary to 
teaching to being an indispensable tool of education. Therefore, IT integrated instruction is distinguished 
from CAI. 



In IT integrated instruction, information technology is an indispensable tool in the teaching environment 
because it is integrated into the curriculum, learning materials, teaching and learning [2]. Moreover, the 
traditional curriculum, materials, and teaching are transformed through the characteristics of information 
technology: the subject-based curriculum and materials become student -based; the teacher-driven teaching 
activities become student-centered. Information technology is integrated when it is used in a seamless 
manner to support and extend curriculum objectives and to engage students in meaningful learning. It is not 
something one does separately; it is part of the daily activities taking place in the classroom [3], 
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igure 1 . The assessment framework for information technology integrated instruction 

Figure 1 depicts the assessment framework of IT integrated instruction. The assessment framework consists 
of two major parts: Kernel Part and Periphery Part. The kernel part primarily assesses the whole teaching 
process. Because the implementation of IT integrated instruction will bring about changes to teaching, the 
aspects to be assessed in this part should include not only the use of IT in teaching but also other 
perspectives of teaching: curricula, learning materials, instruction strategies, learning activities, and 
evaluation. The periphery part primarily assesses the teaching environment, learning resources, information 
equipment, personnel qualities, and administrative as well as professional support. All these factors will 
influence the outcome of teaching. In particular, IT integrated instruction is in need of supportive and 
coordinated environmental conditions. There are many perspectives of the periphery part that are related 
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with IT integrated instruction, and ten of them are carefully identified and included for assessment: teachers, 
students, information specialists, administrators, classroom settings, computer laboratories, campus 
instruction network, Internet, digital materials, and instruction/leaming software. 



3 Assessing the kernel part 

The kernel part refers to the whole teaching process, and Table 1 shows the perspectives and emphases to be 
assessed. The aspects of the kernel part are illustrated in the following paragraph. 



Perspective 

Information technology 
Curricula 

Learning materials 
Instructional strategies 
Learning activities 
Evaluation 



Table 1. Emphases of the kernel part to be assessed 

Emphasis 

The use and role in instruction 

Subject-based separate curricula or Student-centered integrated curricula 
Sequential or problem-based 

Traditional expository approach or constructivist approach 
Teacher-driven or student-centered 

Traditional paper-and-pencil testing or multiple assessment 



3.1 Information Technology 

Information technology may refer to equipment or products, such as computers, network, peripherals, etc. It 
may also refer to the methods or processes in which the equipment of IT is used to help with the solution of 
problems. It is the purpose of implementing IT integrated instruction not only to enable students to use the 
equipment or products of IT but also to use the IT equipment to solve practical problems in learning and life. 

In this perspective, we care about how IT is used in teaching and what role IT plays in teaching. The level of 
the use and role in instruction is developed: 

• Nil (level 0) # IT is not used and plays no part in teaching. 

• Isolation (level 1): IT is used to teach students how to use IT (e.g. keyboarding, drill-and-practice, 
word -processing activities). There is no or little connection between IT and instruction content. 

• Supplement (level 2): Teachers use IT to assist instruction and students use IT to aid learning 
occasionally. IT is viewed as a supplement to existing instructional program. 

• Support (level 3): IT is needed to complete most learning activities. IT serves as a support to 
instruction. 

• Integration (level 4): Students and teachers can use IT in every-day leaming/teaching naturally, 
confidently, and actively. IT is expansively viewed as tool, process, method to find solutions to 
authentic problems in any time anywhere. 

3.2 Curricula 



For elementary and junior high schools, the curricular idea should be life -centered and be in compatible with 
the development process of students’ physical and mental abilities; respect character development, inspire 
individual potential; cultivate civic qualities, respect the value diversified culture system; enhance science 
knowledge and skill, meet the requirements of modem life. The design of curriculum should be based on 
students, on practical experience, and devoted to cultivating the basic abilities required of modem citizens 
[11]. Therefore, the curriculum should be designed as student-centered integrated or interdisciplinary 
curriculum, not subject-centered separate curriculum. 

IT is used as a tool to help students solve the problem. IT literacy should not be taught as an isolated subject, 
nor should activities with IT be isolated from other activities in the classroom [12]. Therefore, Taking the 
students to the computer lab once a week for 40 ninutes is not necessarily integration [3]. The teachers 
should commit to designing student-centered integrated curriculum and integrate IT into the curriculum. 

3 3 Learning Materials 

Textbooks are the main materials for elementary and junior high schools and the primary learning materials 
of students. In traditional education, textbooks were unified, having only one version. They were based on 
subject systems and separate from students’ living experience. Besides, it was difficult to innovate them, 



they could not meet society* s requirements for rapid transformation. In 1996 Taiwan implemented a policy 
which would partially allow publishers to edit and provide textbooks for elementary and junior high school 
so long as they are approved by the Ministry of Education. On February 3, 1999, VIII (2) of National 
Education Act was empowered, which unequivocally directs the full use of ministry -approved textbooks for 
elementary and junior high schools. A new epoch for textbooks was thus heralded in. Teachers should be 
able to exert their professional autonomy, and students should be allowed a flexible, autonomous learning 
leeway. 

The presentation of learning materials should not be limited to static traditional books, but the characteristics 
of computer multi-media should be used to present these materials. Static words and pictures, animated 
pictures and films, voice, acoustic effect and music in combination would make teaching materials lively 
and motivate students to learn. Besides, they can help students to understand abstract concepts or knowledge 
and enhance learning effectiveness. If hyperlink technology is used, nonlinear learning materials can be 
designed so that what students see can be highly individual and not the same. As such, the content of 
learning materials is flexible, adaptive to individual difference, and compatible with the spirit of 
individualized learning. 

In addition to textbooks, there are many resources on the Internet that can be used as learning materials. 
These resources can provide “instant**, “living** information [8]. Teachers not only can search for 
information to be included in teaching materials but also can use the real-time information on the Internet to 
conduct teaching. Students not only can search for information on the Internet but also can conduct 
independent learning any time, any place by using the learning materials on the Internet. 

The use of information technology can make learning materials diversified and lively, make their content 
flexible and integrated with life. Not only can teachers easily motivate students to learn, but students also 
can learn happily in a rich teaching situation. 

3.4 Instruction Strategies 

The teaching strategy of the traditional expository approach is teacher-centered. Students learn what is 
taught by the teacher, but are given a limited room for thinking, discussion, presentation and exploration. 
The teaching effectiveness is ostensibly good, and students’ performance on examination seems impressive. 
Yet this approach contradicts the essence of education. In a series of meaningless learning process, what 
students learn is segmented memory that is extraneous to their experience and cannot be applied in practical 
situations of their daily life. Nor can they enjoy learning. 

Constructive teaching strategy is student-centered. The teacher would first arrange a teaching situation to 
arouse students’ motivation for learning and then would conduce students to explore and think. Through the 
interaction with peers, the students can gradually integrate the new knowledge into their own system of 
knowledge and make it an essential part of this system. By this constructive teaching, students must actively 
learn, while the teacher can only play the role of facilitator, auxiliary, and consultant. 

When students use teaching software and browse for Internet resources, they must explore and think actively 
and construct their own knowledge through the interaction between machine and person and through social 
interaction. Therefore, the teacher is a “coach” for the student rather than a provider of knowledge. Self- 
directed learning is an attainable goal for both the student and teacher when IT is integrated in the various 
content areas [6]. That is, IT integration is most likely to occur in learner-centered classrooms in. which the 
teacher acts as a facilitator [3]. ,**• 

3.5 Learning Activities 

Traditional lecture -based and teacher-driven activities can no longer satisfy the needs of modem education. 
It is not only monotone, also lacks interaction between peers. Learning activities should be student-centered 
so that the learner can actively work to explore knowledge, clarify concepts, and gradually construct his/her 
system of knowledge. In addition, project-based and cooperative learning activities should be adopted to 
allow the learner the opportunity to produce high-level interaction with his/her peers. These activities not 
only can cultivate a respectful, responsible, and confident attitude and the abilities to express, communicate, 
coordinate, think, and create but also can increase learning effectiveness. 

In cooperative learning activities, students can use computer to communicate and discuss, or use a certain 
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support cooperative work software to facilitate collaboration. Finally, multi-media would be used to present 
the learning effectiveness of students. Cooperative learning is not limited in local class. It can also be 
applied across schools, countries, and culture. Therefore, IT enriches the learning activity. 

3.6 Evaluation 

The traditional evaluation approach primarily depends on paper examinations and determines learning 
outcome by the scores on the test sheets. This type of evaluation measures only a dimension of knowledge, 
unable to reflect the wide spectrum of learning process. Future evaluation will become diversified; 
performance evaluation may be conducted along with paper evaluation; students’ self-evaluation, peer 
evaluation and juried evaluation may be conducted along with teacher’ s evaluation; in addition to evaluating 
learning outcome, the learning process should be evaluated; in addition to quantitative evaluation, qualitative 
evaluation should be adopted; in addition to evaluating cognitive domain, the evaluation of affective and 
skill areas should be included. Only such a comprehensive evaluation can reflect the learning process, not 
only be able to understand what the student has learned but also be able to understand how the learning has 
occurred. 

IT integrated instruction is helpful to the implementation of diversified evaluation. For example, the 
electronic portfolio is an ideal means of integrating IT into the instruction. It gives the student and teacher an 
alternative form of assessment. Furthermore, electronic portfolios motivate students to produce quality work, 
and they also increase students’ self-esteem by showcasing their best work [6], 



4 Assessing the periphery part 

The periphery part primarily refers to the surrounding situations. Table 2 shows the perspectives and 
emphases to be assessed. The following illustration is based on perspectives. 

4.1 Teachers 

The teacher is vital in leading teaching activities. Without sufficient information literacy and professional 
ability, he or she cannot apply information technology on teaching, let alone implement IT integrated 
instruction. Regarding professional ability, the teacher should be able to integrate IT, in addition to assessing 
software and digital materials. The attitude is another emphasis of assessment. If the teacher has a positive 
attitude toward computer, he/she can readily introduce and apply computer on teaching; if the teacher can 
accept the change in teaching status and role, the implementation of IT integrated instruction would not 
cause a great impact. 

4.2 Students 

Students are the chief character in education. In teaching, students should take the initiative to construct their 
own knowledge. In implementing IT integrated instruction, students can obtain from the process related 
knowledge and skill and steadily strengthen their information disposition. Gradually students should be able 
to use, naturally and confidently, computer equipment in active learning and to construct their system of 
knowledge. 

4.3 Information specialists 

Teachers are not information specialists. In extensive application of IT to teaching, they will definitely 
encounter many technical problems that can not be solved by them. In this case, information specialists can 
support teachers in solving such problems. It is much easier for information specialists with education 
background to integrate IT with education and guide classroom teachers to implement IT integrated 
instruction. 

4.4 Administrators 

Whether administrators feel important about IT integrated instruction is intimately related with the 
implementation of IT integrated instruction. In addition, if he classroom teacher can gain sufficient 
administrative support, he or she will be more willing to implement IT integrated instruction. 



16 



4.5 Classroom Settings 

Generally speaking, teaching activities are conducted indoors. Therefore, the IT equipment in classroom is 
indispensable to the integration of IT into teaching [14]. The computer and peripherals should not be 
outdated. The operation system and application software installed in the computer should be appropriate for 
the use by students and suit the needs of teaching. Moreover, for a class of more than 10 students, a large 
display device or broadcasting teaching equipment is needed. Finally, it matters whether they are managed 
properly or whether the fair use by students is ensured. 



Table 2, Emphases of the periphery part to be assessed 



Perspective 


Emphasis 


Teachers 


Information literacy and professional competency 

Attitude toward information technologyjind instructional change 


Students 


Information literacy 

Attitude toward information technology 


Information Specialists 


Professional competency 
Support for teacher 


Administrators 


Attitude toward information technology integrated instruction 
Support for teacher 


Classroom Settings 


Number of computers and person-machine ratio 
Grades and fixtures of computer 
Operating system and application software. 
Peripherals (e.g. printer, scanner, digital camera) 
Broadcasting teaching facilities 
Management 


Computer Laboratories 


Number of computer labs, number of computers and person-machine 
ratio 

Grades and fixtures of computer 
Operating system and application software. 

Peripheral (e.g. printer, scanner, digital camera) 

Broadcasting teaching system 
Management 


Campus Instruction Network 


Structure of campus network and network type 

Domain account 

File server and database server 

CD cabinet (perhaps made possible through software simulation) 


Internet 


Method and speed of Internet connection 
Actual connection speed 

Internet server (e.g. web server, proxy server, DNS server, mail server) 
The mechanism to filter out inappropriate information. 


Digital Materials 


Digital materials that can be used on the Internet 
Digital materials created by the teacher 


Instruction/Leaming Software 


Quantity 

Adaptation 



4.6 Computer Laboratories 

In a situation in which IT is integrated into teaching, sometimes it is required that one person have one 
machine. Computer laboratories can justly meet this requirement. Therefore, the management of computer 
laboratories is an important assessment item and can decide whether the computer equipment can 
sufficiently support classroom teachers [14]. Moreover, computer laboratories can also provide the most 
appropriate places for teachers’ advancement and students’ training of information skill. The equipment in 
the computer laboratories should not be outdated. Furthermore, there must be a broadcasting system, 
enabling students to know the whole content of teacher’ s lecture in a ready manner. 

4.7 Campus Instruction Network 

The planning and erection of campus instruction network aims not only to construct an instruction network 
on campus but also to enable every classroom on campus to connect to the Internet through the campus 
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network. After the campus network is erected, File Server and CD cabinet (perhaps made possible through 
software simulation) should be erected, in which the teaching software owned by the school is stored so that 
all the teachers of the school can access to it readily and can apply it to teaching. In addition, the 
establishment and management of network account is equally important, ensuring the safety of information 
[14]. 

4.8 Internet 

There are unlimited, un-exhaustive teaching resources on the Internet. If computers can not be connected to 
the Internet, the application of IT to teaching is compromised. Therefore, it is very important to provide 
information settings of the Internet. In addition, it is needed to erect Internet-related Server, in particular, 
Web Server must be erected so that teachers’ teaching information and the learning outcome of students can 
be stored on it to facilitate examination and simulation by teachers and students. Besides, Internet is full of 
erotic and violent information which is unsuitable to students. It is extremely important to build a 
mechanism to prevent students from seeing those inappropriate content [14]. 

4.9 Digital Materials 

Digital materials may be presented through information equipment and be used in teaching activities. 
Therefore, plentiful digital materials should be able to help integrate IT into teaching of various subjects. 
Therefore many on-line materials on the Internet can be used for teaching purpose. To decrease the amount 
of time required for browsing and facilitate the use of the materials by teachers and students, on-line index 
and search systems are also required. In addition, if on-line materials are not appropriate for teaching needs, 
classroom teachers may create their own materials to be presented on information equipment or use 
materials created by colleague teachers with the approval from the original designer [14]. 

4.10 Instruction/Leaming Software 

Computer Assisted Instruction (CAI) and Computer Assisted Learning (CAL) software is a help to teaching 
and learning. With more software, teachers are better equipped to apply IT to teaching. This software should 
be stored on the CD cabinet or File server on the campus network so that teachers can readily use it 
whenever needed. In addition, if existing teaching software available on campus is evaluated, further 
information can be provided to teachers [14]. 

5 Conclusions 

That teachers and students can extensively use computers for teaching or learning purpose to heighten 
teaching qualities and learning effectiveness is the ultimate goal of the infrastructure construction of 
information education [5]. In other words, integrating computer into teaching of various subjects is the 
ultimate goal of the Ministry of Education in promoting information teaching [13]. What IT integrated 
instruction means is not merely to assist teaching by computer but work to integrate IT into curriculum, 
learning material and learning activities. At this point, the role of teachers begins to transform, from that of a 
main character to that of a support character. Therefore, the implementation of IT integrated instruction not 
only harmonizes with the u ltimate goal of information education but also prompt the reform of education so 
that learning becomes more effective, efficient, and meaningful. 

IT cannot be successfully integrated overnight. It needs to take years to complete the process. The process 
should be carried out in order, stage by stage. Taiwan’ s IT integrated instruction is germinating. The 
assessment framework set forth in this article can be used not only to carry out practical evaluation but also 
serve as reference for development. Teachers’ in-service education, pre -service training, administrative 
support, enriching IT equipment, developing appropriate digital materials and teaching software should be 
taken to strengthen the perspectives of the periphery part and to diversify the surroundings so that teachers 
can realize the benefits brought about by IT on education, Accordingly, teachers can apply IT to teaching, 
gradually infuse IT into learning activities, curricula, learning materials, and adopt student-centered teaching 
strategies and multi- facet evaluation. All this can lead to the fulfillment of the meanings of IT integrated 
instruction. 
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Common authoring shell that enable reuse of teaching materials allows teachers 
to search for existing teaching materials on the web and computer system. The 
authoring shell will give a list of materials that will match the word(s) for 
searching. Sometimes none of the matched items will be reported but sometimes 
hundreds of matched items will be found. It is difficult for teacher to browse all 
matched items before making decision to reuse the material or not under such 
situation. To facilitate the reuse of teaching materials, we expect an authoring 
shell should be more intelligent — able to identify the purpose of the teacher in 
searching for a material. It should provide a list listing closely related materials 
when there is no matching and filter off those materials that are not related to a 
teacher’s intention. In this paper, we shall discuss how it is possible to achieve 
this. 

Keywords: Reuse, authoring, intelligence 



1 Introduction 

There is a lot of authoring tools available that assists teachers in producing online course. Common 
authoring tools being used by educational organizations to produce online course/lesson are Authorware, 
Toolbook, WebCT, BlackBoard, COCA and HyperCard ... etc. Most of these tools can help a teacher to 
create and manage web-based courses even though the teacher may not have any programming or web-based 
design experience. These tools are not domain specific and therefore can be used in various knowledge 
domains. Usually these tools do not explicitly specify the pedagogy or the teaching strategies. Thus, a 
teacher at authoring will not be bounded by specific pedagogy or teaching strategy in using these tools. But 
this also means that a teacher must plan and implement the pedagogy and teaching strategy by 
himself/herself before using the tool. Even though most of these tools allow a teacher to search and retrieve 
teaching materials, they do not actively support the reuse of teaching materials over multiple teaching 
strategies by giving suggestions to teacher. 

IAS, Intelligent Authoring Shell, is a generic authoring shell prototype. Our initiative in building IAS is to 
enable reuse of teaching materials for multiple teaching strategies. By using IAS, the production of online 
adaptive course with multiple teaching strategies can be more cost-effective. It is named with “intelligent” 
because it will suggest closely related reusable materials to a teacher at authoring. 



2 Hypothesis for reusing teaching materials 

There are two basic ways in tackling reuse of teaching materials. One of the ways is based on ontology. In 
1996, Mizoguchi and his fellows discuss using task ontology in building Intelligent Tutoring System where 
“Task ontology is a system/theory of vocabulary for describing inherent problem solving structure of all the 
existing tasks domain-independently” [12]. Laresgoiti [11] also examine the use of ontologies as “vehicles 
for reuse”. They use application ontology, domain ontology, basic technical ontology and generic ontology 
to describe the physical world of education. Similarly, Murray [13] has also discussed using special purpose 
ontologies over object to describe pedagogical knowledge. In 1998, Mizoguchi and his fellows Chen [2] 
have extended the use of task ontology into domain ontology, teaching strategy ontology, learner model 




20 



ontology and interface ontology into their SmartTrainer authoring tool. Using ontology has the advantages 
on re-engineering the structure of knowledge in education and each knowledge domain. This will facilitate 
the use of information technology onto education. However, using ontology in authoring has a potential 
problem — most users, teachers are not familiar with ontology. Unless the operation of ontology is 
transparent to teacher at authoring, it is reasonable that teachers may be reluctantly to use such authoring 
tool. Using ontology aims to standardise the terms to be used so that everyone using ontology will have the 
same understanding on the term being used. However, in education, teachers are encouraged to provide 
innovative teaching in order to promote teaching and student’s learning. Therefore, there exists problem 
between standardisation and innovation. 

Another way to solve material reuse is by using a database repository to store teaching materials information. 
Sarti and Marcke [14] have discussed using database repository method in storing information about 
“Domain”, “Structuring”, “Instructional” and “Presentational” perspective inside DISCourse. Cybulski and 
Linden [3] also suggest using repository method to store the information about various kind of “artefact”. 
They have identified some attributes for the artefact: artefact number, URL, type, label, content description, 
keywords, file type, file size and preview [4][5] in building their MATE tutoring system. Using a database 
has the advantages that teaching materials can be decoupled from the teaching strategies. Besides, it allows 
flexible courseware configuration over LAN and web. Since there are many good off-the-shelf database 
tools available, relatively it is much easier to store, retrieve and maintain teaching materials within the 
system. The disadvantage in using database are the additional cost in setting up and maintaining a database 
system and inherit the limitations from using database softwares. 

Instead of using ontology, we have decided to use database system method to store useful information. 
Unlike methods used in MATE, we use learning objectives as the key to reuse on teaching materials. Most 
teachers are familiar with the concept of learning objective and there is a rich educational research literature 
surrounding them. Even though we know that there are still controversial arguments over using of learning 
objectives, we believe that there are more pros than cons in using learning objectives, particularly under the 
criteria of “effective learning” -- student using minimum time and/or effort to produce maximum learning 
outcome. 

Basic learning objective has two main components: an object and a keyword , the letter drawn from a 
learning objective. An object is a concept, or skill, or entity that is described in a learning objective. For 
example, in a learning objective “Define Newton ’s First Law ”, Newton’s First Law is the object. Keyword is 
the verb being using in the learning objective. Refer to previous example, “ Define ” is the keyword of the 
learning objective. There has been a lot of educational research on keyword (also named as learning 
objective taxonomy). If each teaching material is tagged with at least one learning objective, then from the 
object, we can search for the existence of a related teaching material. Also, from the keyword , we can know 
the purpose of that teaching material. In our IAS, teaching material is a multimedia file corresponding to the 
“atomic artifacts” as defined by Cybulski et al. 

Besides using learning objective over reuse at teaching material level, we can also extend the usage of 
learning objective to learning activity level and lesson level. Learning activities in IAS are activities such as 
introduction, exercise and conclusion that are defined inside a normal lesson structure. Lessons in IAS are 
set of learning activities that are organized for particular teaching strategy such as lecturing, case study, drill 
and practice ...etc. The fundamental difference between our IAS and Cybulski ’s MATE system is; in MATE; 
learning objective is just a statement indicating the usage of teaching materials. In our IAS, learning 
objective is the key for us to build “ domain ”, “ structuring ”, indicator for usage, and “g/ue” to link teaching 
strategies and other educational theories into tutoring systems. 



3 Design of IAS 

In our IAS, we have built following modules: keyword (taxonomy) module, strategy design module and 
course design module. Reuse of teaching materials occurs in course design module. Keyword module and 
strategy design module provide the basic environment that help to provide intelligent suggestions to a 
teacher at authoring. Figure 1 shows the structure of our IAS. 
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Figure 1 Structure of IAS 



3.1 Keyword (Learning objective taxonomy) module 

We have built this module to facilitate entering of learning objective taxonomy ( keyword) into our IAS 
database. In building this module, we have used Bloom’s taxonomy [1] as reference as it is a well-known 
taxonomy being used in education. In general, learning objectives [6] can be grouped as general objectives 
and specific objectives. General objective is composed of a set of specific objectives. Each specific objective 
can be composed of a set of sub-specific objectives or directly related to a lesson or an activity. Bloom has 
classified learning objective taxonomies into three different domains: cognitive domain, affective domain 
and psychomotor skill domain. Under each domain, taxonomies are further classified into different levels. 
For example, within cognitive domain, taxonomies are classified into knowledge , comprehension , 
application , analysis , synthesis and evaluation. These different levels form the class hierarchy with 
knowledge be the lowest class while evaluation is the upmost class. 

In building this module, we are aware that Bloom’s taxonomy is not the only taxonomy available. There are 
Harrow’s taxonomy [7] on psychomotor skill, Krathwohol’s taxonomy [10] on affective domain, Steinaker’s 
taxonomy on experiential learning etc. Therefore, this module allows teacher to enter sets of taxonomy from 
other educational theory school such as experiential learning or “learn by doing”. This feature will allow 
educationists to research and create their own set of taxonomy such as task ontology. We have also added a 
comment field for each keyword so that the definition of each keyword can be added. This will not only help 
teachers to arrive a common understanding of the keyword, but also in learning keywords from different 
educational theory school. Figure 2 shows the data hierarchy being used in this module. 



Taxonomy Domain Class Keyword 




Complexity 
of learning 
increase on 
moving 
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towards , 
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Figure 2 Bloom’s hierarchy of learning objectives on cognitive domain 

3.2 Strategy design module 

From education, there are many teaching strategies such as lecturing, feedback lecturing, concept attainment, 
drill and practice, role-play, gaming, debate etc. Therefore, it is impossible to pre-design all teaching 
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strategies into our IAS. Besides, as mention before, teachers are encouraged to create teaching strategy or 
modify existing strategy in order to provide innovative teaching. Therefore, it is not reasonable if an 
authoring shell cannot allow a teacher to create his/her own teaching strategy. In IAS, we have a strategy 
design module that allow teachers to create their own teaching strategies. Using the strategy design module, 
we have pre-designed some teaching strategies such as lecturing, concept attainment for the purpose of 
demonstration and testing. 

In IAS, a teaching strategy is composed of a set of activities with different activity type. Each activity type 
may consist of sub-activity type. For example, a lesson using concept attainment [8] as its teaching strategy 
usually contains introduction, example, counter-example, analysis and conclusion. A lesson using lecturing 
as its teaching strategy contains introduction, presenting new knowledge, example(s), exercise(s) and 
conclusion. Example is composed of sub-activities: description, question, answer and explanation [9]. 
Therefore, in designing a teaching strategy, a teacher will be asked to construct the structure for the teaching 
strategy by entering and arranging the activity types in order. Figure 3 shows a teaching strategy structure 
for lecturing. 



Strategy Activity Sub-activities 




Figure 3 A teaching strategy structure for lecturing 



In designing the structure of a teaching strategy, the author can enter information or rationale about using 
this teaching strategy. This will help other teachers in leaming/knowing how to use the teaching strategy 
“properly”. Author of teaching strategy is also asked to relate his/her teaching strategy to the domain and 
class of taxonomy that is unlikely applicable. With this information available, IAS can give suggestion to 
teacher what kind of teaching strategy are available and possibly applicable according to the keyword being 
used in learning objective. 

3.3 Course design module 

In designing a course, a teacher will be involved in designing the curriculum for the course, designing the 
lesson and the learning activities within each lesson. Since IAS is just an authoring shell, we will not 
consider the design of teaching materials. Authors can use most off-the- shelve multimedia authoring tools to 
create those teaching materials. In IAS, course design module is the module where most of the reuse of 
teaching materials takes place. 

/ 

3.3.1 Designing the curriculum 

In designing the curriculum, a teacher is required to construct a hierarchy of learning objectives for the 
course. A teacher will enter details for a learning objective including the keyword and object. 

To enter a learning objective, a teacher will be requested to enter the object of the learning objective. With 
the object defined, a list of existing learning objectives with the same object will be displayed for selection. 
If a teacher want to put in a new learning objective, then a teacher will need to specify the keyword. Along 
with the keyword, the taxonomy, domain and class will be identified. Instead of keying in the keyword, a 
teacher may select keyword start from selecting the taxonomy, then domain and class. Information about the 
keyword will be displayed as required. 

In entering a learning objective, it is possible for IAS to provide some basic logical checking over the 
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sequence of learning objective using the keyword class hierarchy. For example, it is not logical to arrange 
“apply for-loop” before “define for-loop” because, according to Bloom’s taxonomy, “apply” is a keyword in 
class application while “define” is a keyword in class knowledge. Higher order classes usually require lower 
order classes as prerequisite. An illogical order of learning objective hierarchy will increase student’s 
difficulty in learning. Therefore, such a checking feature can help to improve quality of a course. 

After the learning objective hierarchy is built, an object hierarchy is also constructed using the objects from 
each learning objective. With this object hierarchy, we can identify the relation between each object. This 
relation will be useful in deciding which teaching materials may be appropriate for reuse. Besides, this will 
also assist us in searching and retrieving materials for the object. 

3.3.2 Designing the lesson 

In IAS, each lesson belongs to only one specific learning objective and each lesson has only one teaching 
strategy. However, a learning objective may have multiple lessons, hence multiple teaching strategies. With 
only one specific learning objective per lesson, it will be easy to replace a lesson with another lesson that has 
the same learning objective, thus provide multiple teaching strategy to student at learning. For example, if 
learning objective “Define for-loop” has two lessons designed using lecturing and concept attainment 
individually, then a student can decide to learn the learning objective using either lecturing or concept 
attainment or even both. Single learning objective per lesson also has the benefit that any modification of a 
lesson will not affect the other learning objectives. 

In creating a lesson, a list of existing teaching strategy for the same learning objective will be available to 
teacher for selection. Besides, that, another list of appropriate strategy (with no activities defined) will also 
be available for a teacher to create his/her new lesson. This appropriate strategy list is generated according to 
the unlikely appropriate strategy for the class of domain keywords as mentioned in the “Strategy design 
module”. For example, in knowing that the lesson is about learning a concept, IAS will give suggestion to 
the author indicating lecturing and concept attainment is recommended. Since IAS only provides suggestion 
rather than making the decision for the author, the author still can select other “less appropriate” teaching 
strategies available. With this feature, a teacher is allowed to use innovative teaching in preparing online 
courses. 

3.3.3 Designing the learning activities 

After a lesson is created, the author will follow the structure of teaching strategy to design learning activities. 
A learning activity may have sub-activities. For example, an example contains sub-activities description, 
question, answer and explanation. For activity that does not have sub-activities, author will be asked to enter 
the link (URL of the teaching material) to the activity. 

In designing the learning activities for a lesson, basically it will use the information about the object and the 
keyword. From the preliminary analysis on reuse [9], we know that teaching materials in activities such as 
example and exercise are highly reusable. Therefore, in suggesting reusable learning activities, IAS will 
include activity type as a factor for consideration. The suggestion criteria available are based on activity type, 
keyword, object and learning objective (keyword + object). Suggestion criteria based on activity type will 
use activity type as filtering element. For example, if the activity at authoring is an example, then IAS will 
only list those activities that are examples only. Suggestion criteria based on keyword will list materials will 
the same keyword. However, this will give a large number of suggestions that are irrelevant due to the 
irrelevant object. Suggestion based on object only will give better results than using activity or keyword only 
because it always return closely related materials with same object. However, there are situations where. IAS 
can not find the exact matching of keyword. Under this situation, IAS will give suggestion according to the 
“distance” from the object to other object on the object hierarchy as defined from the curriculum designed 
module. This will enable the most relevant activity/material will always be suggested. Suggestion criteria 
based on learning objective (keyword + object) will not only list materials/activities that have the exact 
matching on learning objective but also the activities/materials in the order of their relation in learning 
objective hierarchy (curriculum). This will reduce author’s time in searching for reusable teaching materials, 
particularly when the database has thousands of teaching materials available. 

4 Conclusion 

An authoring shell that supports reuse of teaching material should not only allow the author to search for the 
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existence of the material and retrieve it if available. An authoring shell can be more “intelligent” by 
providing suggestions to author at authoring. Even if the searching cannot find the exact matching, an 
intelligent authoring shell should be able to produce a list of materials that are closely related to the 
searching. Our IAS is design not only to enable material reuse but also to assist author in reusing teaching 
material. 

In our IAS, we have included a Keyword Module and a Strategy Design Module to provide the background 
for the intelligence. In the Course Design Module, an author can design the course curriculum, lesson within 
the course and activities for each lesson. IAS will provide intelligent suggestions to the author at each stage 
of design. 

There are still a lot of works that we are going to do in order to improve the performance and intelligence of 
IAS. For example, we are going to find out whether there is an optimal suggestion criterion. Besides, we are 
also investigating the possibility for the IAS to continuously optimise its suggestions by itself according to a 
teacher's selections at authoring and students responses at learning. We know that Keyword Module and 
Strategy Design Modules cannot provide the complete background for the intelligence to be required. We 
still need to explore other modules such as pre-requisite knowledge module into consideration to enhance 
the performance of IAS. 
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With the rapid growth and transfusion of information technology into 
community, the Internet offers a possible new education dimension. It provides a 
vast and integrated resource for individuals by network connecting and web 
interaction without time and location limitations. This paper presents an 
educational web system of integrated route information service (IRIS) that 
recommends and guides students to learn in-depth information of popular tourist 
spots and to travel those spots via the multimodal public transportation systems. 

As a result, the issues of environmental protection in terms of air pollution and 
traffic jam are overcome by making use of the proposed system, and the learner 
is also able to further understand local cultures and transportation information of 
his living area. The proposed IRIS educational service is categorized as a 
combination of transmission and consultation typology based on the 
controllability of information providing and distributing. The design of the 
system and its efficient algorithms for providing the service through the Internet 
will be discussed within the framework of the typology. 
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1 Introduction 

Traffic congestion and air pollution have become major headache for most large cities around the globe. 
Millions of people waste their time in jammed traffic and keep making air polluted by exhaust emission 
everyday. Driving in jammed traffic can also make people very annoyed and tired, inevitably reducing 
people's productivity and devastating state of mind when they finally arrive at their destination. As a result, 
educating people to utilize public transportation systems has become a target for all modem countries, and 
the Internet is one of the best approaches to convey the information and to achieve the goal. 

The congestion problem can be tackled from both the demand and the supply sides. From the supply side, 
one may build more roads to increase the total capacity for traffic. This approach, however, may not be 
practical for many countries and definitely will ravage the natural environment. Land is a kind of limited 
resource, so we cannot construct more roads at the speed that people can buy more cars. ^ 

There are several conceivable strategies to cope with the traffic congestion and air pollution problems from 
the demand side. For instance, we may reduce the traffic volume and air pollution by legislatively banning a 
portion of registered vehicles from entering cities on specific days as what Singapore employs or 
aggravating car taxes for the aged cars as Japan does. Another strategy to reduce the traffic demand is to 
encourage people to commute by public transportation systems. Convenient public transportation systems 
provide not only an economical way for people to commute but also a chance for us to alleviate the air 
pollution problem that results from excessive use of passenger vehicles. 

A popular way to encourage and educate people to use public transportation systems is to provide complete 
route information and to improve the predictability of travel times in an efficient and effective approach. 
Most noticeable measure is the introduction of bus priority routes in many cities [2]. By granting buses 
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exclusive rights to run on chosen routes, people can estimate the travel times on buses more precisely, 
thereby offering an incentive for people to take advantage of buses. Typical multimodal public transportation 
systems provide a complex route network for people to move around the cities. The route network covers the 
served area such that people can virtually walk to their destinations within minutes after departing from a 
nearby bus station. Complex route network, however, is very difficult for people to figure out what routes of 
buses they should take to get to their destinations. Taking the public transportation systems in the Taipei 
metropolitan area for example, more than 200 bus routes serve the metropolis, and the number of bus routes 
is still growing. A railroad route crawls through the metropolis, and five subway routes connect very busy 
area in the metropolis. Searching the best way, or even just a good way, to travel via such multimodal public 
transportation systems can be very challenging to ordinary people. 

Providing route information is certainly an important step toward promoting the use of public transportation 
systems. People will not use the transportation systems effectively unless they have a good idea of the 
service provided by the systems, and people will stop using public transportation systems if they cannot use 
the systems effectively. We are motivated by this observation to build an information service that can help 
people find out their ways in such multimodal public transportation systems in order to solve the problems 
of environmental protection. 

To this aim, we build a web service that provides Integrated Route Information Service (IRIS) to commuters 
and travelers. We collect route information of buses, subways, and trains that serve the Taipei metropolitan 
area, and maintain the collected data with a standard database management tool. Users of IRIS can request 
recommendations through Internet for how to use public transportation to travel from one location to another 
in the metropolis. Using informed-search methods [3], IRIS searches for paths and recommends selected 
paths to users via web connection in a real time manner. The recommendations may provide one or more 
ways to travel between the chosen locations, each of which may require users to transfer among different 
modes of public transportation systems, e.g., from buses to buses, from buses to subways, and from subways 
to buses. 

IRIS system is considered as a travelers* guiding center that provides the information of sight-seeing spots 
and complete travel advisories in transportation. The techniques we employ to build the IRIS for Taipei city 
can be applied to provide similar services in any other metropolis. In a longer run, we can expand the 
database of IRIS to make IRIS a part of the intelligent transportation systems that will modernize the 
transportation systems in Taiwan or other countries[4]. 



2 Taipei public transportation systems 

There are three basic types of public transportation in Taipei: trains, buses, and rapid transit system. Taiwan 
Railway Administration (TRA) operates traditional train system that runs through the heart of the metropolis, 
and there are a half dozen of stops in the area. We can obtain train schedule from both the booklet published 
by the TRA and its web site [10]. The Taiwan High Speed Rail Corporation [9] and the central government 
plan to build a high-speed train system that will connect major metropolis on the island, but the system will 
not start to operate in the near term. 

Several bus companies, including publicly and privately owned ones, jointly operate the Taipei bus systems. 
These companies serve more than 200 bus routes in the city, and these routes altogether have about 2500 
stops distinctively. Various sources of bus-route information are available nowadays. Traditionally, people 
obtain bus-route information either from booklets or from signboards at bus stops that pontain such 
information. Those booklets can be purchased from any book retailer island wide. Recently, the Internet 
brings us a convenient media for distributing bus-route information. The Taipei city government provides 
on-line bus-route information on its web site [6,7], and there are personal web sites that provide free route 
information as well. 

The Taipei Rapid Transit Corporation (TRTC) operates the Taipei Rapid Transit system (TRT). People may 
find information related to the TRT from both brochures published by the TRTC and its web site [8]. The 
TRT system consists of five different color-coded routes and there are four special locations where people 
may transfer between subways, traditional trains, and buses. They include Panchiao, Lungshan Temple, 
Sungshan and the Taipei Main Station that lie at the intersection of the red and the blue lines. 

A traveler obtains the complete information through Internet for each type of public transportation 
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respectively, however, he may not be able to conclude the best vehicle combination for a specified 
original/destination pair for himself. Therefore, he may result in giving up the usage of public transportation 
and drive his own vehicle instead, and perhaps the air condition will be devastated in a further step. In the 
following section, we introduce the methodology of collecting public transportation information for each 
type and propose an efficient and intelligent algorithm for an optimal combination via Internet access in real 
time. A traveler can access the web site to learn the best way to arrive his destination by public 
transportation system. 



3 Route-information database 

3.1 The raw data 

Information from multiple sources can cause data inconsistency problem. Both TRA and TRTC provide 
authoritative information about the routes that are under their control. Since TRT and traditional trains 
typically do not change routes, it is easy to take care of route information about these systems. Bus routes, in 
contrast, can change from time to time due to a variety of reasons such as road construction and 
demonstration events, etc.. Also, route-information sources, both web sites and books, typically do not 
reflect the route change in a timely manner. Our experience indicates that chance is high that bus-route 
information provided by different sources does not completely agree with each other. 

Fortunately, bus companies generally do not change bus routes randomly. Usually, bus stops are relocated to 
nearby locations. Providing information that becomes out-of-date because of such minor route changes 
might cause nuisance, but the routing information should remain valuable. Therefore, we adopt the route 
information that is recorded in a booklet [5] in building our route-information database. 

We maintain route information about buses, TRT and trains in a similar way. Tables for all routes contain the 
departure, intermediate, and destination locations in the metropolis. Routes served by different systems are 
marked by an attribute, type , in the route table. Each route table contains the type, the route number, and 
names of the departure, intermediate, and destination stops. The intermediate stops are ordered according to 
when a passenger may pass the stop from the departure stop, so a stop will have an order attribute in the 
database. Since regular routes that connect a particular pair of locations, A and B, typically provide round 
trip service, we treat routes that go from A to B and that go from B to A as two different routes. 

We construct stop tables based on these route tables for convenience of reference. Each stop table contains 
what routes are available, the order of the stop on these routes, and if TRT and traditional trains are available 
at the stop. For further processing of the raw database, we also include information about the longitude and 
latitude for each stop in the database, and which will play an important role for geometrical information 
retrieval or WAP application in the near future. 

3.2 A hierarchical map 

We preprocess the raw database for the route-planning algorithm. Let X and Y be different stops, we use the 
following terms in the discussion. 

(1) Stop route and stop size'. In practice, a stop is a small area in the metropolis, and typically one or more 

routes may allow passengers to get on and off at a stop. The stop route, denoted SRftC), is the set of the 
routes that serve the stop X, and the stop size of a stop X, denoted SS(X), is the number oifroutes that 
serve the stop X. Notice that the routes can be either bus, TRT, or traditional train routes. 

(2) Stop distance : The stop distance between X and Y, denoted DS(X,Y), is the minimal number of 

intermediate stops that we would pass if we travel from X to Y via public transportation. 

(3) Geographic distance’. The geographic distance between X and Y, denoted DG(X,Y ), is the Euclidean 

distance between X and Y. 

As we inspect the stop tables, it becomes clear that some stops are relatively more convenient in terms of 
ease of transferring between routes and are denoted as hubs. These hubs should be seriously considered as 
transfer centers when there is no direct route connecting the traveler’s origin and destination. The Taipei 
Main Station, for instance, provides a great chance for people to transfer to routes that eventually lead to 
their destinations. 
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To take advantage of this observation, we introduce an attribute in the stop tables to reflect that if a stop is 
considered as a hub according to its easiness of transferring. To this end, we consider stops that have TRT 
and train services as hubs. Currently, we set the hub attribute if the stop size of the bus stop is greater or 
equal to 15. Therefore, in addition to TRT and train stops, we have about 70 hubs for Taipei area in our 
database. 

We annotate tables for each stop X with information about the nearest hubs , denoted NH(X). A nearest hub 
of a stop X is a hub that can be reached by taking a particular route in SR(X) from X. Clearly, if a stop is 
itself a hub, then the stop is also the nearest hub for all routes in SR(X). Except this trivial case, we define 
the nearest hub B of a stop X on a specific route based on the following criteria that are listed in the 
descending priority. 

(1) DS(H,X) > DS(B,X) for any hub H 

(2) If DS(H1,X)=DS(H2,X), then B=H1 when SS(H1)>SS(H2) and B=H2 when SS(H2)>SS(H1) 

(3) When there are still multiple candidate, we let B=H1 if DG(H2,X) > DG(H1,X). 

Usually, the first criterion suffices for determining a nearest hub on a route from X because most routes pass 
a hub or two. When a route from X does not pass any hub, transferring to other routes will be considered, 
and this is when we may need these criteria. 

To speed up the route-planning algorithm, we have done some preprocessing on the raw database. We add 
into each stop table the routing information between the stop and its nearest hubs. This extra information 
will be helpful when we need to transfer via nearest hubs. Also, we compute routing information between 
any TRT and traditional train stations, and save the results in the route-information database. This 
information can be very useful at runtime, but does not require a large space to save, so it is worthwhile to 
carry out this preprocessing offline. 

Technically speaking, the route map in our database has two levels. The lower level consists of all stops, and 
the higher level consists of only hubs in the metropolis. To utilize this hierarchical organization, we compute 
the best ways to travel from hubs to hubs offline, and save such information in the database. We will see 
how the route-planning algorithm harnesses such an effort next. 



4 The route-planning algorithm 

Although the route -planning algorithm does not necessarily provide the fastest solution for traveling from 
origins to destinations, the algorithm does attempt to do so based on a few heuristics. First, the algorithm 
strongly biases for the use of TRT and traditional trains.* It typically takes less time to commute by TRT or 
trains than by buses between two locations. The algorithm also prefers solutions that require less transfer and 
less number of intermediate stops. Therefore, the algorithm prefers a routing method that needs only one 
transfer to one that needs two transfers. In the algorithm we define and compute at runtime the set of stops 
RftQ that are directly reachable from a stop X via SR(X). 

The algorithm requires information about the desired origin and destination. At this moment, both origin and 
destination are names of stops, so the algorithm does not have to worry about cases where no connecting 
routes are available. There is an easy way to expand the algorithm to allow street addresses as input for 
origin and destination since our stop tables already have the information about the longitude aqcUatitude of 
each stop. The skeleton of the algorithm follows. 

Algorithm RoutePlanning(origin O, destination D, route-info database) 

1 . If O is equal to D, there is no need to commute. 

2. Direct link: If the intersection of SR(0) and SR(D) is not empty, there are ways to go from O to D 

directly, and the algorithm recommends such direct paths. If both O and D are TRT stops, recommend 
the TRT first. 

3. One transfer: If D is in R(R(0)) y we may go from O to D with one transfer, and the algorithm 

recommends these paths. 



For simplicity, we will not distinguish between TRT and traditional trains henceforth. 
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4. Transfer via hubs: Recommend the path from O to its nearest hubs NH(0), from NH(O) to NH(D), and 

then from NH(D) to D . 

5. Sorting and listing: Sort and list the possible route combinations based on stop distance DS(0 , D) and 

stop size of transferring node SS(X). 

Whenever the algorithm finds multiple solutions to the query, it orders the solutions by the total number of 
intermediate stops. This designed decision is based on the assumption that more intermediate stops may lead 
to a longer total travel time. This principal may apply to solutions found at any step. For instances, at the 
second step, the algorithm may find multiple bus routes that directly connect O and D, and these routes will 
be listed in the order of increasing number of intermediate stops between O and D. 

When there is no direct way to commute from O to D, the algorithm continues to the third step. This step 
dictates that commuting from O to D needs one transfer if there is at least a stop Z in the set of R(0), and D 
is in the set of R(Z). We can thus infer that it takes just one transfer to go from O to D if the intersection of 
R(0) and R(D ) is nonempty. This method may be a good heuristic, but is not flawless. Recall that R(Y) 
represents the set of stops that are directly reachable from Y. The fact that the intersection of R(0) and R(D) 
is nonempty implies that there is a route from O to an intermediate stop Z and another route from D to Z. 
There is no warrantee to find direct routing method from Z to D, which is what we really need. There must 
be a stop X that can be directly reached by a route, say RA , from O, and must also be directly reachable by 
another route, say RB, from X to D. The algorithm will frequently find many related one-transfer solutions, 
and some of them appear to be better than others. Consider commuting from A to B in the example shown in 
Figure 1. Since there is no direct way to go to B from A, we must transfer at Tl, T2, or T3. This might 
appear to be a contrived example, but it is not. In reality, bus routes often pass a portion of business districts, 
so they tend to have a few consecutive common stops. Examining these three alternatives for transfer, we see 
that the traveler can only transfer from R1 to R2 at either Tl or T2. The traveler will have two choices, R2 
and R3, if s/he transfers at T3. Therefore, T3 appears to be a better choice for transfer because it provides 
better chance for the traveler to catch the next bus sooner. Due to this observation, our algorithm lists as 
better alternatives those stops that have larger stop sizes for transfer. 



When there are neither direct nor one-transfer routes that connect A and B, the algorithm reaches the fourth 
step. At this step, the algorithm applies the information contained in the database about nearest hubs of A 
and B to find desired solutions. Recall that we have computed the best routes from A to its nearest hubs 
NH(A) and the best routes from NH(A) to any other hubs including the nearest hubs NH(B) of B. The 
remaining task is to find a route from NH(B) to B. This is not a difficult task in practice since we can find at 
least a stop X in NH(B) such that B is in R(X). 

Figure 2 illustrates that, to commute from A to B, we 
can commute from A to one of its nearest hubs, HI, 
then to one of B’s nearest hub, H3, and finally to B. 

In this case, the route -in formation database will 
contain the best way to commute from HI to H3 is 
via Rl, and the best way to commute from HI to H3 
is via Rl and then transfer onto R6 at D. 



5 User interface 




Figure 1: T3 is a better transfer location 



In addition to provide intelligent public 
transportation information, the IRIS system 
collects general travelling information at Taipei, 
and it also provides information about popular 
tourist spots including natural beauty, cultures 
(museum/memorial/temples and historical 
sites/festival/folks art /aboriginal art/towns and 
city), amusement parks, shopping and 
entertainment, food and beverages, real-time 
weather report, real-time traffic surveillance, and 
related travel services. All these services and 
introduction are categorized and constructed in 
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the way of friendly and graceful page design. Unlike the consultation typology of public transportation 
querying system, the design pattern of travelling information leans to one-way communication of media 
technology. The activities of users are pure reception, and the pattern of communication is categorized as a 
transmission typology. 

To obtain the quickest route information of an area or spot, users can click one of following items listed to 
acquire the correlative information: route information (complete stop information for each bus/TRITTRA 
route), stop information (complete bus-route information for each bus route), local area route 
information(complete transportation information for each local area), popular tourist spot route 
information(complete transportation information for each tourist spot), original/destination Integrated Route 
Information(complete route information from original to destination). All these items are designed as a 
format of consultation typology and users have to make a request to the information providing center for 
specific message to be delivered. For examples, in the application of original/destination Integrated Route 
Information, users are allowed to type partial text of original/destination information through the main 
interface, and the IRIS system will base on the input contents and response an interface with option selection 
buttons for complete inputs. After analyzing the selected original/destination location buttons, IRIS system 
will verify the data, execute the matching programs, and show the results by listing route numbers and 
transfer stops in different color text, and the system also provides the map information of selected route and 
stops respectively. The querying processes and results are displayed in Figure 3. Listing the results with text 
may allow the system to provide more alternative routes at a time, but showing the routes on the map gives 
travelers a clearer picture of the recommended routes. The system is welcome to access and evaluate through 
the web address http://iris.cs.ntou.edu.tw/ . 



6 Conclusions 

The proposed educational service is categorized as a combination of transmission and consultation typology. 
The part of general travelling information of IRIS system which provides and controls information 
distribution serves as a transmission communication pattern. When the learner makes a request and system 
provides an integrated information immediately, the system performs as a consultation typology since the 
information is produced by the system but the learner retains control over what and when the information is 
distributed. The objective of the proposed web services is to provide an integrate and optimal solution for 
tourists, however, there are still some future work left to be completed in this project. As we have reported, 
the IRIS system prioritizes alternative solutions based on heuristics that take into account the number of 
intermediate stops and times of transfers. Although these may arguably be good rules of thumb for selecting 
fast routes, a traveler might prefer routes that minimize the total distance that s/he would travel. To provide 
such alternative, the system would have to know the actual distance traveled by the bus for moving between 
two consecutive stops. In a longer run, the IRIS system may allow travelers to determine if the system 
should prioritize alternative solutions by the monetary cost, frequency of buses, and even predicted travel 
time. 

The path-planning algorithm reported in Section 4 prefers the travelers to transfer at a stop that has a larger 
stop size. This designed decision is based on easy of transferring to another bus. Travelers may have other 
concerns though. A traveler might prefer to transfer at previous stops because it is more likely to have a seat 
if one can get on the bus earlier. A flexible system should allow travelers to choose her/his way for 
prioritizing the alternative solutions. 

In addition to the future work, we have started to build a path-planning algorithijV-'based on 
uninformed-search algorithms[3]. However, these algorithms might not provide satisfactory performance for 
ordinary shortest-path applications when the network is large. Nevertheless, these algorithms might provide 
satisfactory performance for constrained shortest-path applications in a median-sized city like the one we are 
tackling. Considering the fact that uninformed-search algorithms are far easier to construct than the 
in formed- search algorithms, uninformed-search algorithms might be a viable way to the bus scheduling 
problems. We will report findings of our exploration in an extended version of this paper. 
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Figure 3: Examples of original/destination Integrated Route Information 
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The purpose of this study was to clarify the cognitive effects of collaborative 
learning on Junior high school students' Logo programming. Two experiments 
were implemented: Experiment 1 was an analysis of the relationships between 
interaction in pair activities and students' reflection. The effects of pair learning 
on students' promoting abilities of programming were analyzed in Experiment2. 

As the results of Experiment 1, students’ self-monitoring and self-control were 
supplemented each other through the interaction. Results of Experiment 2 
suggested that the effect of collaborative learning on students' programming 
abilities were developments of debugging ability against syntax error and coding 
ability of lower students, which was obtained the cognitive strategies for task 
division through the interaction. 

Keywords: Collaborative Learning, Junior High School Students, Cognitive 
Effects, Logo Programming 

1 Introduction 

In Japan, education about computer programming was placed in Fundamentals of Information of Industrial 
Arts at junior high school level from 1989. From 2002, programming, sensing and control will be placed in 
Information and Computer of Technology as an elective learning content ( Course of Study published in 
1998)[5]. Many technology teachers in Japan thought that teaching programming was not only for 
professional higher education. They didn't made points of understanding the function of software upon a 
computer system, but acquiring the problem solving skills through the programming activities. 

Historically, many researchers suggested that one of the methods for acquiring the problem solving skills 
was collaborative learning. It was necessary for students to communicate and interact with someone who 
had same goal in collaborative environment (Deutsch 1 949)[ 1 ]. In the recent past, it was supported that the 
experiences of solving the problem through the interaction made the processes of planning and decision 
making clearly each other, and would promote their self-control and self-monitoring when they ^yould solve 
another problem all by themselves (SATOU 1996)[3]. In the case of learning about programming, KAGE 
(1997) suggested that 12-year old pupils showed vigorous verbal interaction, which led them to more 
sophisticated problem solving [4]. 

From these findings, it was predicted that acquiring the problem solving skills brought to promote students' 
programming abilities as a result of cognitive effects of collaboration. 

The purpose of this study was to clarify the cognitive effects of collaborative learning on students' 
programming. For this purpose, two experiments by using Logo programming (Japanese Edition) were 
implemented. The aim of Experiment I was to clarify the relationships between interaction of collaborative 
learning processes and learners' reflection. The effects of collaborative learning on students' promoting 
abilities of programming were analyzed in Experiment2. 
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2 Methods 



2.1 Experimentl 

2.1.1 Subjects 

Twelve 3rd grader Jr. high school students (6 males and 6 females) were divided into 6 pairs. 

2.1.2 Instruments 

"The Reflection Scale of Thinking Process on Computer Programming: RSTC M (MORIYAMA et al 1996) [2] 
and the modified LUTE (Link-UniT-Element) model (MORIMOTO et al 1997) [6] were used for measuring 
the level of reflection and analyzing the interaction, respectively. The RSTC was constructed from 4 factors 
as in Fig.l. Factorl was the reflection of understanding the problems and enterprising how to make the 
program adequately. Factor2 was the reflection of designing the program and coding. Factor3 was the 
reflection of self-monitoring on each parts of the program on the local level. Factor4 was the reflection of 
self-monitoring on the whole program and renewal of problem representation. 



Factorl (6 items) 


Factor2 (6 items) 


Factor3 (5 items) 


Factor! (3 items) 


Semantic understanding of the problem 
Imaging the command and grammar 
Comprehending the image of program 
Rhetorical understanding of the program 
Seeking the semantically-related knowledge 
Seeking the rhetorically-related skill 


Setting up the keywords 

Division of the program 

Setting up the functional unit 

Connecting the functional unit 

Coding the functional unit 

Checking the sequences of each commands 


Predicting the result of running 
Testing walk through the list 
Checking the clerical error 
Checking the syntax error 
Checking the logical error 


Analyzing the bug 

Renewal ofproblem representation 

Seeking the bug 



Fig.l The Reflection Scale of Thinking Process on Computer Programming: RSTC 



The modified LUTE model was shown in Fig.2. There were categories for analyzing interaction of 
collaborative learning in this model, and this model had three abstract levels: element, unit and link level. 
The items of element level were categories for functions of protocols. The unit and link level categories 
were for phases and contexts in their programming activities. 



Element Level (5 categories) 



Proposed 
Agreement 
Quest ion 
Opposition 

Supplementary explanation 



Unit Level (6 categories) 



Phase of Analysis 
Phase of Plan 
Phase of operation 
Phase of Edit 

Phase of Checking the program list 
Phase of Checking the result of running 



Link Level (6 categories) 



Link for Formation of plan 
Link for Modification of plan 
Link for Implementation of plan 
Link for Check of Implementation 
Link for renewal of plan 
Link for renewal of implement ation 



Fig.2 The modified LUTE (Link-UniT-Element) model 



2.1.3 Procedures 

Subjects were asked to make the Logo program which draw the "House" constructed from triangular shapes, 
square patterns, circles and lines in pair. Their activities were recorded on a VTR. After they finished the 
task, they answered RSTC individually. Their protocols were extracted from the VTR and were categorized 
by using modified LUTE model. The level of reflection and the relative interaction in the collaborating pair 
were analyzed by ANOVA on mean scores of frequencies of link level categories and Coefficient of 
Correlation (r) between the RSTC scores and frequencies of the element and unit level categories. 
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2.2 Experiment2 

2.2.1 Subjects 

Sixty 3rd grader junior high school students (30 males and 30 females) were divided into 2 groups learning 
Logo programming. One was collaborative learning group (pair), and the other was individually learning 
group. 

2.2.2 Instruments 

The achievement tests and the RSTC were prepared. The achievement tests included both the coding test and 
the debug test. The coding test asked to make a program drawing " Scarecrow " on an answer sheet. The 
debug test asked to find three types of error, clerical error, syntax error, logical error from the program list 
which drew " Spaceship ", 

2.2.3 Procedures 

The procedure was shown in Fig.3. At first, subjects had a coding test which draws the easy "flag" as a 
pre-test. Next, subjects were asked to make the program, which draws the "House" such as Experiment 1 and 
answered RSTC in every group as a middle-test. Finally, they had the achievement tests and answered RSTC 
individually as post-tests. The effects of collaborative learning on students' promoting abilities of 
programming were analyzed by using ANOVA and Coefficient of Correlation (r) between the RSTC scores 
and the Achievement tests’ scores. 




Fig.3 The procedure of Experiment! 



3 Results and Discussion 

3.1 Experimentl: Students' Reflections and Collaborative Programming 

3.1.1 Contexts of Collaboration in the Pair Activities 

There were differences of period of keyboard operation time in pair activities. In this analysis, long-operated 
learners were called Learner A , and the others (short-operated) were called Learner B. Mean scores of 
frequencies of link level categories were shown in Table. 1. 



Table. 1 Frequencies of link level categories 



Link Level Categories 




Mean Score (S.D.) 




ANOVA 


Learner A to B 


Learner B to A 


Learner A to A 


Learner B to B 


Link for Formation of plan 


1.75 (1.04) 


2.00(1.77) 


1.63(1.41) 


3.50(2.73) 


n.s. 


Link for Modification of plan 


3.35(2.12) 


2.50(1.93) 


0.25(0.46) 


0.13(0.35) 


F(3,24)=8.397, p<.01 


Link for Implementation of plan 


1.88(2.70) 


15.63(5.80) 


5.88(3.40) 


2.75(2.49) 


F(3,24)=2 1.732, p<.01 


Link for Check of Implementation 


3.75(1.49) 


1.13(1.36) 


1.00(1.07) 


0.13(0.35) 


F(3,24)=13.055, p<.01 


Link for renewal of plan 


0.38(0.52) 


0.38(0.74) 


0.13(0.35) 


0.63(0.52) 


n.s. 


Link for renewal of implementation 


0.63(0.92) 


1.25(1.28) 


0.25(0.46) 


0.00(0.00) 


n.s. 



Results from Two-way Repeated Measures ANOVA showed that there were significant main effects of 
Links for Implementation of Plan from Learner B to A [F(3,24)=21 .732, p<.0 1 ], and Links for Check of 
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Implementation from A to B [F(3,24)= 13.055, p<01]. Also, Links for Modification of Plan with interaction 
{B to A and A to B) were increased than that of individually links (A to A and B to B) [F(3,24)=8.397, p<.01]. 
These data indicated that the role of operation {Learner A) and the role of planning {Learner B) were shared 
in pair activities. However, it was suggested that consensus decision making through the interaction was 
important for building up their programming plans. 

3.1.2 The Relationships between the Interactions and the Reflections 

Coefficient of Correlation (r) between the RSTC scores and frequencies of element level categories were 
shown in Table. 2. According to these data, when Learner A (operator) proposed something to operate, the 
reflection of designing the program (Factor2) was promoted in own thinking process [r=0.88, p<.01]. 
However, when Learner B (planner) proposed, the reflection of self-monitoring on each parts of the program 
(Factor3) was promoted in Learner A' s thinking process[r=0,88, p< 01]. Furthermore, opposition by Learner 
A correlated the reflection of self-monitoring (Factor3) in Learner Z?'s[r=0.71, p<.05]. Also, Learner A' s 
reflection of designing (Factor2) was promoted by the opposition of Learner Z?[r=0.77, p< 05]. These results 
indicated that the verbal communications on their interaction brought out their self-monitoring and 
self-control each other. 



Table.2 Coefficient of Correlation (r) between the RSTC scores and frequencies of element level categories 



Flpmpnt T 1 P 


Factor 1 


Factor2 


Factor3 


Factor4 






Learner A Learner B 


Learner A Learner B 


Learner A Learner B 


Learner A Learner B 


Proposed 


Learner A 


0.41 


0.26 


0.88” 


0.41 


0.52 


0.13 


0.06 


0.06 


Learner B 


0.04 


0.45 


0.37 


0.25 


0.8$** 


-0.33 


0.50 


0.31 


Agreement 


Learner A 


0.00 


-0.10 


0.31 


-0.34 


0.52 


-0.66 


0.26 


-0.20 


Learner B 


0.44 


0.35 


0.67 


0.55 


0.28 


0.08 


0.36 


0.07 


Question 


Learner A 


-0.32 


0.56 


0.27 


0.28 


0.56 


-0.11 


0.78* 


0.23 


Learner B 


0.40 


-0.52 


0.56 


-0.29 


0.00 


0.02 


0.03 


-0.31 


Opposition 


Learner A 


0.03 


-0.30 


0.27 


0.08 


0.13 


0.71* 


0.42 


0.19 


Learner B 


0.12 


0.19 


0.77* 


0.23 


0.52 


0.27 


0.68 


0.00 


Supplementary explanation 


Learner A 


0.72* 


-0.35 


0.61 


0.13 


0.03 


0.40 


-0.15 


-0.01 


Learner B 


-0.01 


-0.34 


0.32 


-0.13 


0.14 


0.40 


0.45 


0.08 



n=8, df=6, **p<.01, *p<.05 



In addition, Coefficient of Correlation (r) between the RSTC scores and frequencies of unit level categories 
showed that, operation by Learner B as a planner conduced to Learner A's self monitoring on whole 
program[r=0.85, p<.01]. Also, task analysis by Learner A as an operator encouraged Learner B's designing 
of the program[r=0.75, p<.05]. It was evident that one’s reflective thinking was precipitated by the 
observation of the other's behavior which was supposed to be his own behavior 

These results of Experiment 1 suggested that students' meta-cognition (self-monitoring and self-control) were 
supplemented each other through the interaction of collaborative pair learning. 

3.2 Experiment2: Effects on students' promoting abilities of programming 

3.2.1 Acquisitions of Programming Abilities 

In the pre-test, there are not significant differences between the pair learning group and the ^dividually 
learning group [F(l,56)=0.65, Students who could get high scores were called higher students and the 
others were called lower students in this analysis (both 50% and n=30). In the middle-test, mean score of 
RSTC in the pair learning group (0.77) was higher than that in the individually learning group (0.56) 
[F(l,56)=32.40, p<.01]. This result supported findings of Experiment 1 because collaborative pair learning 
could promote students' reflections of thinking processes. 

Mean scores of debug test were shown in Fig.4. Results from the ANOVA showed that the debugging scores 
of syntax error in the pair learning group was higher than that in the individually learning group 
[F(l,56)=4.75, p<.05]. But, there were not significant differences on the debugging scores of clerical and 
logical errors [F(l,56)=2.06 and F(l,56)=0.89, both n.^.]. These results indicated that collaborative pair 
learning could form students' debugging abilities against syntax errors, at least. 
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Mean scores of coding test were shown in Fig.5. The result from the Two-way Repeated Measures of 
ANOVA showed that there was significant interaction between High-Low student condition and 
pair-individually group condition [F(l,56)=10.46, p<.01]. Furthermore, from the results of Simple Main 
Effects Tests, the score of lower students in the pair learning group was promoted to the same level as higher 
students in both groups [F( 1,56)= 12.56, p<.01]. These results indicated that the coding abilities of 
Low-Ability students could be pulled up through the interaction with High-Ability students. 



Score 




*p<05 

Individually Learning Group Pair Learning Group 

Fig.4 Mean scores of debug test (syntax error) 



Score 




Individually Learning Group Pair Learning Group 



3.2.2 Acquisitions of Cognitive Strategies 

Coefficient of Correlation (r) between the RSTC scores and the achievement tests were shown in Table3. 
According to these data, there were significant correlation between the coding test and the RSTC items: 
“Division of the program"(r=0.3 1, p<.05), “Coding the functional unit“(r=0.41, p<.01), “Connecting the 
functional units“(/^0.40, p<.01) and “Selecting the commands for each functional units“(r=0.40, p<.01) . 
Also, There were significant correlation between the debug test and the RSTC items: “Division of the 
program“(r=0.29, p<.05), “Checking the sequences of each commands“(r=0.33, p<.01). It was indicated that 
promoting these reflections were responsible for the development of the programming abilities. Furthermore, 
these items suggested the reflections of cognitive strategies for task division. 



Table 3 Coefficient of Correlation (r) between the RSTC scores and the achievement tests 

Achievement Tests 



Items of RSTC — — — — 

Coding Test 


Debug Test 


Division of the program 


0.31 


* 


0.29 * 


Coding the functional unit 


0.41 


** 




Connecting the functional units 


0.40 


** 




Selecting the commands for each functional units 


0.40 


** 




Checking the sequences of each commands 






0.33 



**p<.01, *p<05 



Results of Experiment! suggested that the effect of collaborative learning on students’ programming abilities 
were developments of debugging ability against syntax error and coding ability of lower students, which was 
obtained the cognitive strategies for task division through the interaction. 



4 Conclusion 

In this study, it was clarified that students’ meta-cognition and cognitive strategies could be acquired through 
the collaborative learning at junior high school level, also that the RSTC was useful for measuring students’ 
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reflections in their programming activities. These findings will contribute to the researches of developments 
of collaborative learning systems. 

For the future, learning processes and cognitive effects of more widely collaborative learning environment, 
for example, distributed programming by using CSCL system or long distance education for programming 
by using Internet, must be analyzed. 
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There are two types virtual school administration systems, web-based or voice- 
based, which are currently used by students. They are systems with different 
access mechanisms but same business logic, and require two times of resources 
for development and maintenance. Whenever the business logic of the systems 
changes, both of the systems need to be implemented. As the wireless 
communication grows more popular, the school has been considering adding a 
wireless interface to the system. However, with current architecture, the only 
way to add a wireless application protocol (WAP)-based system is to implement 
an additional system from scratch. Since the voice-based system and the web- 
based system have the same business logic, they can be integrated into one. We 
can dedicate an application server for the business logic, which interacts with the 
web-based interface and the voice-activated interface with a set of application 
programming interface (API). With the extraction of the business logic and the 
business logic API, developers for the voice-activated interface and the web- 
based interface can implement the interfaces without specific knowledge of the 
business logic of the system. With this design and architecture, the system can 
be further expanded to support a WAP -based interface and other interfaces easily. 

Keywords: Internet, wireless, virtual school, heterogeneous 



1 Introduction 

The Internet is widely used for school education, especially virtual school education [2] [3] [4]. The 
advantage of the Internet is its capability of supporting multimedia and its attractiveness to the user. For the 
virtual school education, the students study via the Internet. They do not have to be in the classrooms of a 
school and can leam at anywhere at anytime. However computers and communication networks are needed 
to support virtual education through the Internet. The cost of the computers and setting up the 
communication networks is very expensive. Thus, the systems are not available everywhere. Furthermore, 
system interfaces must be developed in order to allow the users to access the computers and the networks. 
The purpose of these system interfaces is to provide an easier way for the students to access the systems and 
to allow the students to interact with the instructors real-time. Those systems interface do n of Heed to be 
attractive and colorful since its main goal is to provide a mechanism for the students to access information 
real-time. For a web-based system, the homepage can be design in a way to reduce the network traffic and 
system load. However, not every student can access the computers and the networks due to his financial 
situation or the load of the system. For the students who cannot access the computers and the networks, the 
telephone (the voice activated based interface) provides another popular access media. Therefore there are 
needs for systems to support both telephone (voice- based) and web browser (web-based) interfaces [1]. The 
web-based system is more visual and more user friendly, however, the voice-based system is more 
convenient, more affordable, and requires no hardware investment from the students. As the technology 
evolves, the wireless communication is gradually taking over the traditional wire line communication. To 
support the wireless communication the system will need to be expanded to support the wireless application 
protocol (WAP)-based interface [10]. 
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Originally, a couple of the school administration systems we had can be accessed via a regular telephone or 
via a web browser but not both. They were basically two different systems, though they support the same 
business logic. Both of them have their own user interface and system logic and were designed, 
implemented, and maintained separately. To support them two sets of resources are needed. The original 
system architecture is shown in Figure 1. Developers for both of the systems handle both the business logic’s 
and the user interface’s design and implementation. Whenever the business rule changes, both of the systems 
need to be modified and updated. It is very costly and difficult to keep both of the system consistent. 




Figure 1 . Logic view of voice system and web system 

To reduce the maintenance cost of the two systems and to make them easier to be upgraded and expanded, 
we have proposed to integrate the two systems by extracting the business logic module out of them and 
migrate it into an application server. The remaining of the systems is migrated into a web server and a voice 
server respectively. By doing this, we dramatically reduced the cost of maintaining the system. After the 
architecture change, whenever there is a business change, only the application server is affected. We reduced 
the maintenance cost by 50%. No more concerns about the consistency of the systems. With the 
modification of the system architecture, we make it more scalable and expandable. The system can be easily 
expanded to support other access media without making changes to the application server. For example, to 
support a WAP-based interface, a WAP server can be easily introduced and integrated into the modified 
system architecture. 



2 System Architecture and Implementation ^ 

2.1 Architecture 

The administration system is an N-tiered system. 

• Data Services Tier: The database services and implementations. 

• Business Logic Tier: The business rule of the system. 

• Translation Tier: Translate the I/O between application server and gateway server. For the voice-based 
system, the gateway server is the voice server. The purpose of the voice server is to translate PSTN and 
HTTP between application server and usual telephone. For the web-based system, the translation tier is 
transparent; it does not do anything. For the WAP-based system, the WAP Gateway is the gateway 
server. The purpose of WAP Gateway is to translate the WSP/WTP and HTTP between WAP telephone 
and web server. 
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• Presentation (UI) Tier: The input and output of the web-based system is HTML. The input and output of 
the voice system is the key press and voice of usual telephone. The input and output of the WAP-based 
system is WML [10]. 
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Figure 2. System Architecture 

In the Architecture, the application server is the most important part. The application server needs to process 
business logic and interact with voice server, web server, and WAP Gateway. Because the protocol between 
the application server and the voice server and the WAP Gateway is HTTP protocol, we can set the 
application server and the web server in the same machine. The developers of the application server are 
more responsible, because they must handle business rule, HTML and WML. The developers of the other 
systems implement User Interface and do not have the knowledge of business rule of the system, because the 
developers of the application server handle the business rule. The application server sends different output 
format to different systems by parameters. Under the Architecture, after building the web system, the other 
systems are easily to build. 



2.2 Architecture of the Voice System 

Because taking business logic out of the voice system, the function of voice system is coherent. It translates 
the output of web server to telephone. The output format of web server is HTML. So the voice server has to 
simulate to web browser, shown as in Figure 3. 
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Figure 3. The communication methods are between voice server and web server. 
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3 Case Study 



The Enrollment System of the Tamkang University [7] is designed and implemented following the 
architecture of this paper, shown as in figure 4. The system has been deployed and used by thousands of 
concurrent users [8]. 

3.1 Hardware Structure 

We used thirteen Pentium based servers to implement the system. Six of them are used as the web servers. 
One machine is used as the UNIX Gateway. One server is used as the alert and automating email server. 
Four voice servers are used to support the voice activation. Finally, all student enrolment information is 
stored in one database server. The network hardware are two 1 00 MB/sec switch hub. 




OS: Microsoft NT4.0 is used for the web servers, voice servers, and the alert and automating email server. 
Free BSD 3.0 is used for the UNIX Gateway [8]. 

Web server: Microsoft IIS 4.0. 

Database: Microsoft SQL Server 6.5. 

3.3 Load Balancing and Scalability 

To make the system suitable for all schools, we also took into considerations of the cost of hardware and the 
scalability of the system. A set of low-end servers can be grouped together to replace a high-end server[6]. 
To achieve this, a DNS server is needed for the load balancing work. The simple round robin methodology is 
used for the load balancing. With the current flexible four-tiered architecture, servers can be added info the 
system to share the performance load whenever the system load is heavy[9]. 

3.4. Security 

Two security strategies are used to increase security: 

1. Packet filter: It only allows IP packets through port 80 to access the web server, the packets of the other 
ports can not pass through. The web system can avoid being attacked by the other machines. 

2. Supports multi -protocol: TCP/IP protocol is used between the web server and outside systems. IPX 
protocol is used between the web server and the database server. The web server should be hacked, the 
database server is kept away Internet and the database is still safe. 
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3.5. Network Management and Monitoring 

The alert system has the following features: 

1. Monitoring the system: It sends to keep-alive message to web servers, voice servers, and database servers 

in every period. 

2. Network management system: Checks network traffic between web servers, voice servers and database 

server. 

3. Auto Backup the data of database server. 

3.6. User Interface Design 

One of the most important criteria of the virtual school administration system is to let students access and 
retrieve correct information real-time. The user interface must be simple to reduce network traffic and 
system download time. The homepages for the web system and WAP are simple and straightforward to 
improve system performance. The look and feel of the WAP homepage depends on the WAP telephone the 
user users. An Ericsson r320 model WAP homepage is shown here as a sample WAP homepage. We can 
compare the home pages for the web system and WAP system. 
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Figure 5. The display of the homepage of WAP-based system 



3.7 Lag statistics and analysis 

Duration of enrollment period, the system generates the log automatically everyday for statistics and 
analysis, as shown in Table 1. 
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Table 1 . Tamkang University Daily Enrollment Statistics 
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By comparison, the load of the web system is much heavier than the load of voice system. Since the voice 
system has 32 telephone lines, it can only support 32 concurrent users. In the peak hour of the enrollment 
(the first hour of each grade enrollment), the load of the web server is high. 

We expect the voice system and the WAP system to be fully loaded during the peak hour. A dedicated 
business logic-processing server is used for the voice system and the WAP system. Since the telephone lines 
of the voice system and the WAP system are limited (up to 32 lines), a dedicated web server for the business 
logic processing of the voice system and the WAP system is sufficient. 



4 Conclusions and future development 

The development and maintenance resource of the heterogeneous systems depends on how many access 
media. The more access media, the more resource it needs. My proposal has the following advantages: 

• Resource Reducing: Because the business logic is centered, heterogeneous systems need one business- 
logic process only, the resource of development and maintenance is less than usual systems. 

• Expandability: With the N-tiered system architecture design, the business logic system was designed 
and implemented to support different UI systems. Different UI access method can be easily added into 
the system. 

In the system, the application server interacts with voice server and WAP Gateway on HTTP protocol., so 
the application server must have functions of the web server. We can develop a new structure of the 
application server for voice-based system and WAP-based system, and the application server interacts with 
the voice server and WAP Gateway on TCP/IP. 
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1 Introduction 

The official memorandum is a very important document that offers a decision path of something in the most 
organizations. In general, the executed policy usually needs agreement of decision-maker through official 
memorandum. All the official memorandums are traditionally passed by one-by-one human delivery from 
faculties to managers in an organization. It may results in the lower performance of administrative. Even 
though some administrative operations are via e-mail or other approach. It have some limitations, such as the 
official documents usually need the signature of decision-maker, it is not ease to overcome via the e-mail. 

In order to have a speedy the administrative operation environment, especially the official memorandum 
delivery, we design and implement a WWW-based Official Memorandum System in a school. That is a 
WWW application without any novel theory and technique. We apply the existing techniques used in the 
WWW environment to accomplish the application. 

Obviously, the system is based on the client-server model. Implementing the system has some existing 
techniques can be used, such as CGI, Java/Servlet [3], Java/CORBA [3], Lotus’ InterNotes [1] product uses 
CGI mechanisms to allow Web browser access to documents and forms managed by the Notes Server. 
Documents to be placed on the Web are translated by a program to HTML. These documents and forms are 
accessed through a standard HTTP server as though they were normal HTML documents. Java is a portable 
object-oriented language, and also a good platform for writing client/server web-based applications. Servlets 
are secure protocol and platform-independent server side web-enabled software components, written in Java. 
Java/CORBA has a clear advantage over CGI solution, such as flexibility, maintainability, and 
responsiveness etc. 

Security issue in the system will be taken care by using traditional approaches. There are two secure 
mechanisms will be used: one is account/password, the other is the firewall. First one can prevent 
non-authority user log-in into system and disrupt the system. All the general users must apply for an account 
excepting the chief of department. And the system will force all users to change the password periodically. 
This mechanism can avoid internal hackers. Second one is to avoid external hackers who intrude into system 
for non-authority accessing. Few hackers, of course, can intrude into and disrupt the system. Some 
approaches can be used for enhancing the security of information, such as data compression/decompression 
before accessing to/from database and checking the data consistency of duplicated database periodically. All 
of them are the future works 

Fault tolerance is in order to enhance the reliability of system. In fault -tolerance community, many 
approaches have been proposed to enhance the data reliability [4,5]. The approach in the system is database 
replication. We use warm stand-by prim ary /backup scheme to improve the system availability. Many issues 
in the data replication that have to be guaranteed are employed like the [5], These issues are such as 
idempotent operation, data consistency, and recovery. Because the system is a three-tier scheme, all 
operations supporting fault-tolerance are implemented in the core of the system. This feature can also 
prevent the database crash during the formal execution phase. 

A complex system has to be manageable in an easy way. In order to enhance the system flexibility, a 
web-based management tools should be implemented. System manager can add and remove user easily. In 
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addition, system manager can also maintain the database, such as record manipulation, in an easy way. 



Many features are described previously. In addition, we will support some important functions shown as 
following: Official documents writing, Official documents progression tracking, Auto -delivery, Automatic 
signing, Urgent document notification. 

2 Design and Implementation 



According to the described above, we design the system architecture like as Figure 1. The architecture is 
simple and complete. The system includes an Official Memorandum System and a replicated database. The 
system will receive requests from clients. For security issue, we add a firewall in the front of web server. All 
the requests must be checked by the firewall for ensuring the request is an authority request. In addition, the 
Official Memorandum System is responsible for all the features described above, which include 
fault-tolerance. A replicated database is also included in the system. The database used in the system is the 
SQL database. 




The whole system is implemented and run on the Windows NT 4.0 and SQL server 7.0. The programming 
paradigm is ASP that using VBscript. With the fault-tolerant, the system needs to access primary and 
standby database separately. To guarantee the consistency cf two databases, we apply the traditional 
two-phase commit protocol on the replicated database transaction processing. 

Figure 2 shows the GUI of document reviewing for those chiefs of department. When they login into the 
system, the system will show the urgent document on top of the reviewing page, which indicate these 
documents have to review first. The document reviewing process will sign the signature automatically when 
the process achieved. 
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Figure 3. The GUI of Document Reviewing 

3 Conclusions 

In this paper, we have been stated the design and implementation of a web-based official memorandum 
system. This system can migrate the conventional official memorandum system to network. That is a WWW 
application without any novel theory and technique. We apply the existing techniques used in the WWW 
environment to accomplish the application. In order to avoid the informal accessing to this system, the 
firewall is utilized at the front-end of the system. Besides, the duplicated databases are used in this system to 
prevent the database crash during the formal execution phase. 
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With the rapid growth of both computer technology and the Internet, conventional 
models of testing are gradually being replaced by CAT (Computer Assisted Testing) 
systems. However, the major problem in most CAT systems is the difficulty in 
refreshing and supplying test items. This paper presents a novel network CAT system, 
DIYexamer (Do-It-Yourself Examer). It has three features that differentiate it from 
existing CAT systems: student DIY items, item-bank sharing, and automatic 
assessment of item discriminability. DIYexamer accepts test items contributed form 
teachers as well as students, and allows limited item sharing between item-banks 
possibly maintained by different organizations. An algorithm is applied dynamically 
to assess the discriminability of items in item-banks in order to filter out less qualified 
contributions, hereby assuring the quality of stored items while scaling up the size of 
item-banks. 

Keywords: computer assisted testing, test evaluation, test acquisition, 
discriminability, distant learning 



1 Intorduction 

With the continuing development of computer technology and the Internet, educators now have new alternatives 
for creating, storing, accessing, distributing and sharing learning as well as testing materials. Should testing be 
performed on or learned from computers, and then a computer can best assess the work, Bugbee ( 1 996)[ 1 ]. Hence, 
assessing the learning achievements and attitudes of students via computers or networks becomes a challenging 
task for many educators and researchers. . 

A. Computer-assisted Testing Categories 

Computer-assisted Testing (CAT) or Computer-based Testing (CBT), the use of computers for testing purposes, 
has a history spanning more than twenty years. The documented advantages of computer administered testing 
include reductions of testing time, an increase in test security, provision of instant scoring, and an individualized 
adaptive testing environment [2][3][4][5]. As listed in Table 1, three categories of CAT are currently employed: 
standalone packages, test centers and networked systems. 



- 1 - 
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TABLE 1 : Categories of CAT 





Network 

support 


Item 

generator 


Random item 
selection 


Item 

source 


Item quality 
assessment 


Item-bank 

sharing 


Test result 
analysis 


Standalone 

package 


No 


Built in 
item-bank 


Yes 


Fixed 


No 


No 


Yes 


Test center 


Yes 


Expert 


Yes 


Limited 


Yes 


No 


Yes 


Networked 

system 


Yes 


Built in 
item-bank 


Yes 


Fixed 


No 


No 


Yes 



1) Standalone package: This type of computer software package is typically stored on disks or CD-ROMs. Some 
packages have built-in item-banks, while others require teachers to input test items. These CAT packages 
generally do not have network capabilities. 

2) Test center: The test centers or lab services require dedicated computer terminals for testing purposes. Students 
are required to complete the computer-based tests at the centers. Well-known applications of this type of service 
are Graduate Record Examinations (GRE) and Graduate Management Admission Test (GMAT), as provided 
by Educational Testing Services (ETS) [6]. 

3) Networked system: This enables students to perform an examination through an Internet connection. 
Concurrent testing of multiple users, automatic score calculation, and automatic test result analysis are 
supported by a networked system. The major advantages of networked systems are the convenience of 
examinations and test result calculation. However, the major flaws are the limitation of the amount of items and 
no item discriminability assessment. 

B. Problem Statement 

Regardless of which CAT system is employed, a critical issue in developing CAT is the construction of a test 
item-bank. Traditionally, asking teachers and content experts to submit items generates the item-bank. Three major 
drawbacks of the traditional method can be observed: 

1) Limitation of item amount: Teachers and content experts tend to have similar views on the test subject. That is, 
in a given field vital subject matter might be confined. Therefore, although more teachers and content experts 
are invited to contribute test items, the total number of distinct items remains low. 

2) Passive learning attitude: Students are conventionally excluded from the creation of tests. In a typical 
computer-assisted testing system, teachers generate tests, the system presents test sheets and students then 
complete the tests. That is, within the system of testing, they play a passive role, and are not afforded the 
opportunity to conduct “meta-learning” or “meta-analysis.” 

3) No guarantee on item quality: Permitting students to generate tests may be a possible solution to the 
aforementioned problems. However, this raises a new problem: quality assurance and ensuring that the tests are 
worth storing and used for further tests. Even when the whole item-bank is contributed by teachers and content 
experts, ways to dynamically assess and filter test items are needed. 

The rest of this paper is organized as follows. The three distinct features of DIYexamer are introduced in section 2. 
Section 3 describes how the DIYexamer was implemented and its functionalities for administrators, teachers, and 
students. The discriminability calculation formula is then presented in section 4. Finally, the accuracy of 
discriminability discretion of DIYexamer and conventional methodology are compared through a real-life test in 
section 5. 

2 The Diyexamer Solution 

The DIYexamer[7] is a Web-based multi-server system that allows students to contribute test items, and provides 
an effective means of verifying the discriminability of these items. Three main ideas are as below: 
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Fig 1 : Students generate items into the item-bank 

1) Item DIY by students: DIYexamer allows students to generate test items into the item-banks online as Fig 1 . 
Teachers can query these items generated by students as Fig 2. In addition to rapidly increasing the total 
number of items in an item-bank, this feature also encourages students to develop m eta -learning, i.e. creative 
learning . In order to submit tests, students must thoroughly study the learning materials, develop higher-level 
overviews of the materials, and practice cognitive and creative thinking. 
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Fig 2: Student DIY items as queried by teachers Fig 3: Average item discriminabilities of item-banks 



2) Assessment of item discriminability: DIYexamer provides an item-discriminability assessment method to 
ensure the quality of the stored items. In addition to ensuring the internal consistency of existing test items, this 
method also continuously and dynamically screens additional new items in the item-bank/Fig 3 shows the 
average item discriminabilities of several item-banks. 

3) Item-bank sharing: DIYexamer, a scalable multi-server system, connects many item-banks stored in different 
servers. Therefore, via the Internet, more items can be accessed and shared. The sharing is limited and 
controlled in a sense that a server issues a request, describing the criteria of a test item it requests, to another 
server. A server does not open up its item-bank for unlimited access. 

Additional advantages have been identified and include the facts that since DIYexamer provides a real-time 
on-demand generation of test-sheet function, cheating is avoided. Also, DIYexamer provides an item 
cross-analysis function to which the degree of difficulty for each test as well as the entire test base can be 
accurately measured. 
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3 Diyexamer System Implementation 

A. DIYexamer Network Architecture 

DIYexamer is a WBT (WWW-Based Testing) system. An important feature is the sharing of item-bank via 
network connections. According to Fig 4, several DIYexamer servers form a scalable test union. Therefore, each 
server can access other servers and thus achieve item-bank sharing. A remote server can also join the test union to 
share additional test-bank resources, and leave the test union without affecting other servers. 




B. Internal System Model 

The internal architecture of DIYexamer (Fig 5) is divided into three layers. Interface layer is responsible for 
providing web interface for users. Test Profile Layer (TPL) selects items to form a test sheet, computes scores, and 
calculates the discriminability of selected test items. Test base Sharing Layer (TSL) accesses both local and remote 
databases via a network. Three functions of TSL are listed in Table 2: 




Generate 

Item 


On-Line Test 


Management 






Item 

Selection 


Test Sheet 
Generate 


Test Sheet 
Analysis 


Iti 

Veri 

Z' 


em Discriminability 
fi cation & Assessment 

"s 



TSL 



[Interface 
| Layer 



VTPL 




Fig 5: Structure of DIYexamer 
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TABLE 2: Functions of TSL 



Function 


Description 


Add new items 


New items and corresponding answers are categorized to 
specific chapters and stored in the local item-bank. 


Access local item-bank 


Accessing local while generating test sheets and calculating 
discriminability. 


Connect to remote 
item-bank 


Item-bank sharing through a connection to a remote 
item-bank. 



Environments and development tools used to construct DIYexamer are listed in Table 3. Perl is used to write CGI 
programs to create user interface as homepage. Apache, an open source web server software, is responsible for 
front-end. The back-end, item-bank, is handled by Postgres. 



TABLE 3: Environments and development tools 



Function 


Tools 


HTTP Server 


Apache 1 .3.3 [8] 


Programming Tools 


Perl 5.003 [9] • CGI.pm-2.56 [10] - Gd.pn-1.14[l 1] 


DBMS 


Postgresql 6.5.3 [12] 


O.S. 


Red Hat Linux release 6. 1 [ 1 3] 



C. Functionalities for administrators, teachers and students 

DIYexamer provides a web interface for users to remotely control and operate the system. Three types of users are 
supported: administrators, teachers, and students. Corresponding functionalities are listed in Table 4. 

TABLE 4: Functionalities for different users 





Administrator 


Teacher 


Student 


System and Database 


• Examine system status 

• Join a test union 

• Leave a test union 

• Create personal accounts 

• Create group accounts 

• Modify accounts 

• Modify item-bank 

• Redact course division 

• Backup database 


• React course division 




Item production 




• Create tests sheet 

• Select test items 

• Edit test items 

• Read test items 


• Edit test items 

• Read test items 


Test 


• Network Invigilate 


• Input scores of homework 


• On-line test 


Analysis 


• Analyze tests 

• Analyze test items 


• Analyze tests 

• Analyze test items 

• Analyze subjects and divisions 


• Analyze tests 


Inquiry 




• Inquire tests 

• Inquire test items generated by 
students 

• Inquire students scores 


• Inquire personal scores 

iV 



4 Discriminability Assessment Of Diyexamer 

A. Method of Traditional Discriminability Assessment 

A criterion against which the quality of test items is judged is the assessment of discriminability. An item is 
regarded as with high discriminability when competent students correctly answered it, while less competent 
students incorrectly answered it, and vice versa. When computing item discriminability, those students with 
relatively high and relatively low scores are taken as samples. Those students whose scores fall in middle range not 
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considered. Next, item discriminate lity is computed according to the performance of these sampled students when 
answering each item. 

In the traditional discriminability assessment method[14], those in the top 30% and the bottom 30% rank groups 
are chosen as samples. The top 30% scorers are defined as “high-rank group (H)”, while the bottom 30% scorers 
are defined as “low-rank group (L)” The formula for calculating the discriminability of an item is as follows: 

The number of students in H The number of students in L 

that answered correctly that answered correctly 

Discriminability = — 

The number of students in H The number of students in L 

In the traditional method, two major drawbacks can be observed. The first one has something to do with whether 
the 30% is in terms of count of students or range of scores. The sampled students fall in the top 30% and the bottom 
30% rank groups, i.e. in terms of counts. However, it is possible that these scores differ only slightly from the 
average score especially when scores are not wide-spread distributed, where many scorers should not be 
considered in computing the discriminability. Second, the effect on discriminability assessment by each student in 
either group is assumed to be the same. However, those students that received different scores have different 
degrees of tendency to correctly or incorrectly answer an item. For example, a sampled student who received 97 
points should have higher referential value than a sampled student who received 80 points. 

B. Method for DIYexamer’s Discriminability Assessment 

When selecting sample students, only those whose scores have large gap with the average score should be 
considered. Accordingly, those with the top 30%, in terms of range, scores are defined as “high-score group (H’)”, 
while those with the bottom 30% scores are defined as “low-score group (L’)”. 

To show the different criteria and effects of choosing samples in the traditional method and DIYexamer method, 
Fig.6 depicts the score distribution in a test. In this example, the highest score is 92, the lowest score is 34, and the 
average score is 69. The “high rank score group” and the “low rank score group” are chosen according to these two 
methods. Take student X as an example, the score of X is 66, which differs only 3 points from the average score. 
The associated information of X should have little, if not none, referential value in computing item discriminability. 
However, X is chosen as a sample in the high rank group in the traditional method. This fallacy results from using 
rank group, in terms of count, as the criterion of choosing samples. In DIYexamer, X is not chosen since score 
group, in terms of range, rather than rank group is used. Only those with large gap with the average score are 
chosen as samples. 




Fig 6: Comparison of samples taken in the traditional method and DIYexamer method 

For different samples to have different impacts on discriminability, a referential value with respect to an item is 
generated for each student selected as a sample. We first define the item discriminability as the average of all 
associated referential values, as shown below: 
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Sum of the referential values of sampled students 



Discriminability = 

Number of sampled students 

Since the referential values depend on students’ scores, the referential values are computed according to the ratio 
of correct and incorrect answers of the sampled students. The ratios of correct and incorrect answers are defined as 
follows: 

Number of items answered correctly 

Ratio of correct answer = 

Number of items on the test 

Number of items answered incorrectly 

Ratio of incorrect answer = 

Number of items on the test 



TABLE 5: Principle to compute the referential value of a student with respect to an item 



Student 


Answer 


Item 

discriminability 


Referential valne to compute 
discriminability 


Competent 

(With high ratio of correct answer) 


Correct 


High 


Ratio of correct answer 


Incorrect 


Low 


Ratio of incorrect answer 


Less competent 

(With low ratio of correct answer) 


Correct 


Low 


Ratio of correct answer 


Incorrect 


High 


Ratio of incorrect answer 



According to Table 5, the referential value of a student correctly answered an item is the ratio of correct answer of 
the student. Alternately, the referential value of a student incorrectly answered an item is the ratio of incorrect 
answer of the student. This policy comes from the fact that an item should have increased discriminability if 
correctly answered by a competent student, while rendering decreased discriminability if correctly answered by a 
less competent student. In this way, a competent student contributes large referential value to a correctly answered 
item and small referential value to an incorrectly answered item, and vice versa. 

C. Algorithm for DIYexamer’s Discriminability Assessment 

The test result of a student is used if the score falls in either the high or the low score group. A referential value is 
computed for each item the student answered. The discriminability of an item is the average of all of the associated 
referential values. 

To calculate for each item, information must be recorded in the database. First, the highest and the lowest scores 
(i.e. Gmax and Gmin) of all students who answered the question item are recorded to calculate Gh and Gl. Gh and 
G1 are used as thresholds to determine whether a student is eligible to affect the rating of an item. Second, the 
number of students with referential values (i.e. n) and the sum of referential values (i.e. Accumulator) are recorded. 
The calculation formula and the corresponding definition of used parameters are listed below. Algorithm of 
DIYexamer’s discriminability assessment summarized in Fig 7. 

Accumulator: sum of referential values 

n: number of students with referential value 

T: number of correctly answered questions in this test 

F: number of incorrectly answered questions in this test 

Gmax: highest score of all students answered this question 

Gmin: lowest score of all students answered this question 

Gh: high threshold for ratio of correct answer 

Gl: low threshold for ratio of incorrect answer 

Ans: A Boolean variable indicates whether a student correctly or 

incorrectly answered the question 
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if((T/(T+F)>Gh) or (T/(T+F)<G1)) 

{ 

if (T/(T+F)>Gmax) 

Gmax = T/(T+F) 
else(T/(T +F)<Gmin) 

Gmin = T/(T+F) 

Gh = Gmax-(Gmax-Gmin)*30%; 

G1 = Gmin+(Gmax-Gmin)*30%; 
n = n+ 1 ; 

if (Ans=Correct) 

Accumulator = Accumulator + T/(T+F); 
else (Ans=Wrong) 

Accumulator = Accumulator + F/(T+F); 
Discrimination = Accumulator /n; 

j 

Fig 7: Discriminability assessment algorithm 



5 Evaluation Of The Discriminability Assessment In Diyexamer 

The fairness and performance of DIYexamer was evaluated. We conducted an experiment where 10 students took 
the test on-line using DIYexamer with 10 items. Table 6 summarizes the test results. Fig 8 shows the score 
distribution of the experiment. Discriminability for each item is computed using both the traditional method and 
the DIYexamer method. However, the discriminability originally falls between -1 to 1 using the traditional method, 
while falling between 0 to 1 using the DIYexamer method. To compare these two methods, both two ranges of 
discriminability are then normalized to 0 to 10, as shown in Fig 9. 

According to Fig 9, the item discriminability differs in these two methods because the samples taken are different. 
The low-score group consists of student 1, 2, and 3 by the traditional method, while only 1 and 2 by the DIYexamer 
method. In this case, student 3 got 4 points, which differs from the average score (i.e. 5.2 points) by only 1 .2 points. 
Since student 3 should have little, if not none, impact on the assessment of discriminability, student 3 is in fact not 
a proper sample. 

Observe that, in Table 6 , student 1 who is a less competent student and has incorrectly answered all items except 
item 1, and student 10 who is a very competent student and has incorrectly answered item 1. Thus, item 1 can be 
concluded 4 s of low discriminability. Comparing the assessment results in these two methods, the computed item 
discriminability of item 1 is very low in the DIYexamer method but not as low in the traditional method. 

Comparing item 3 and item 1 in Table 6 , item 3 should have higher discriminability than item 1 because competent 
students tend to answer item 3 correctly and less competent students tend to answer item 3 incorrectly, which is not 
true for item 1. However, item 3 and item 1 have the same discriminability, i.e. 5, by the traditional method. In this 
case, the actual discriminability is more accurately reflected in the DIYexamer method than in the traditional 
method. 



TABLE 6 : Result of the test experiment 





Item 


Item 


Item 


Item 


Item 


Item 


Item 


Item 


Item 


Item 


Number of correct 




1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


answers(score) 


student 1 


1 (correct) 


0 (incorrect) 


0 


0 


0 


0 


0 


0 


0 


0 


1 


student2 


1 


0 


0 


1 


0 


0 


0 


0 


0 


0 


2 


student3 


0 


1 


1 


0 


1 


0 


0 


1 


0 


0 


4 


student4 


1 


1 


0 


1 


0 


0 


0 


1 


0 


1 


5 


student5 


0 


1 


0 


1 


0 


0 


1 


1 


0 


1 


5 


student6 


1 


1 


1 


0 


0 


1 


0 


1 


1 


0 


6 


student7 


0 


1 


1 


1 


1 


0 


0 


1 


1 


0 


6 
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student8 


1 


0 


0 


1 


1 


1 


1 


0 


1 


1 


7 


student9 


1 


0 


0 


0 


1 


1 


1 


1 


1 


1 


7 


student 10 


0 


1 


1 


1 


1 


1 


1 


1 


1 


1 


9 




Fig 9: Comparison of item discriminability 



6 Conclustion 

This paper has presented a novel architecture for a networked CAT system, DIYexamer. It supports item DIY by 
students, item-bank sharing, and item discriminability assessment. 

For discriminability assessment, new calculation formula were proposed. When compared with the traditional 
assessment scheme, the main difference is that the top and the bottom 30% of the score group, in terms of range of 
scores were selected rather than the rank group, in terms of count of students. Thus, item discriminability is more 
accurately reflected particularly when the tested students have close scores. 

Item-bank sharing and item DIY by students has increased both the amount and the variety of questions in 
item-banks. Item DIY by students promotes creative learning within students, while automatic discriminability 
assessment assures better quality than traditional CAT systems. 

A questionnaire was used to survey subjective attitudes of students about DIYexamer. As shown in Table 7, the 
outcome revealed that most students were interested in item DIY. 

TABLE 7: DIYexamer questionnaire results: percentage and the number of students in parentheses of each question 



Question 


Strongly agree 


Agree 


No opinion 


Disagree 


Strongly 

disagree 


Item DIY is interesting. 


12.3 (7) 


43.9 (25) 


21.1 (12) 


15.8(9) 


7.0(4) 


Item DIY is fanciful. 


19.5(10) 


49.1 (28) 


21.1 (12) 


10.5 (6) 


1.8(1) 


I am curious about the testing result of my DIY item. 


26.3(15) 


59.6 (34) 


10.5 (6) 


3.5 (2); 


0.0(0) 


I learned a lot when creating items. 


12.3 (7) 


47.4 (27) 


.22.8(13) 


17.5 (10) 


0.0(0) 


I am curious about the teacher’s opinion about my 
DIY item. 


22.8(13) 


50.9 (29) 


22.8(13) 


1.8 (1) 


1.8(1) 


I am curious about other students’ opinions about my 
DIY item. 


15.8(9) 


56.1 (32) 


21.1 (12) 


7.0 (4) 


0.0 (0) 


I studied harder to prepare item DIY. 


10.5 (6) 


54.4(31) 


21.1 (12) 


14.0 (8) 


0.0 (0) 


Judging the difficulties of my DIY items is easy. 


40.4 (23) 


38.6 (22) 


14.0 (8) 


7.0(4) 


0.0 (0) 


Judging the fitness of my DIY items is difficult. 


36.8 (21) 


49.1 (28) 


8.8 (5) 


5.3(3) 


0.0(0) 


Item DIY by students comes from the laziness of 
teachers. 


7.0 (4) 


12.3 (7) 


43.9 (25) 


33.3 (19) 


3.5 (2) 


If possible, I hope such item DIY mode through the 
whole course can replace conventional testing. 


1.8(1) 


10.5(6) 


35.1 (20) 


38.6 (22) 


14.0(8) 
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Items generated by students are easier than by the 
teacher. 


7.0(4) 


36.8(21) 


28.1(16) 


24.6 (14) 


3.5 (2) 


I knew more about the testing material after item 
DIY procedure. 


8.8(5) 


50.9 (29) 


22.8(13) 


15.8(9) 


1.8(1) 



The technique proposed herein is useful in general tuition not only to improve the quality of test items and fairness; 
but also to save time from generating questions and computing scores. We recommend that DIYexamer be 
popularized to schools. 
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There are great potentials for the use of computers in the enhancement of 
teaching and learning in secondary schools, but in some subject areas, the 
realisation of these potentials is critically limited by the lack of appropriate 
educational software. Custom development of this kind of software is often not 
a viable alternative, since such a task is well known to be non-trivial and time- 
consuming that is frequently beyond the capacity of individual secondary school 
teachers. As computer science researchers and educators, we are aware that vast 
amount of teaching resources are freely available on the Internet. Such 
resources are often used by tertiary educators for enriching their teaching, but 
largely under-utilised by secondary school teachers. This paper reports our 
experience in the design and delivery of a short course which aims at refreshing 
practising secondary school computer teachers with updated knowledge on 
teaching and learning with computers. We describe how we achieve our goals of 
providing practical assistance to computer teachers by empowering them to 
effectively exploit Internet resources for use in their schools. Our approach is 
enabling in that it fosters participants’ lifelong learning beyond the contents of 
the present course, and is applicable to a broader context than ours. 

Keywords: Teacher education, lifelong learning, program visualisation, 
algorithm animation 



1 Introduction 

For a long time, educators and computer scientists have been exploring the use of computers in education [9]. 
The rapid drop in hardware price and the tremendous improvement in computing power in recent years have 
rendered computers more affordable to schools, teachers and students. Hardware is no longer the bottleneck 
that hinders the integration of information technology (IT) into the school curriculum. There are 
increasingly great potentials for using computers to enhance teaching and learning at all levels p£education. 
In some subject areas, however, the realisation of these potentials is severely limited by the lack: of 
appropriate educational software. 

The development of good quality CAI software is well known to be a non-trivial and time-consuming task 
that calls for the combined expertise of programmers, experienced educators, graphics/multimedia designers, 
and others [10]. Such a task is often beyond the capacity of individual teachers in primary and secondary 
schools, due to their limited time, technical expertise and perhaps monetary resources. More fundamentally, 
it would not be realistic to require every teacher to develop their own CAI software from scratch for use. 
This is even true for most university educators. As Resmer [13] argues, “if every professor in a university 
had to write their own textbook, typeset it, print it, publish it, bind it, and distribute it before their students 
could use it, [textbooks] would not be a viable learning resource”. Likewise, for widespread and effective 
use of computers in education, there is a need for teachers to be well informed of the source of available 
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educational software. 



The Internet promises to be a source of many valuable teaching resources that are frequently available freely 
or at affordable costs. There are many advantages of exploiting Internet resources for use in teaching. Apart 
from cost savings, software tools on the Internet are more likely to be kept up-to-date as technology 
advances, and their evaluation versions could be put to trial use before making actual purchases. 

By nature of their work, many university educators are accustomed to the exploitation of Internet resources 
for both research and teaching purposes [14]. In contrast, these resources have largely been under-utilised 
by secondary school teachers due to various reasons. Firstly, many teachers are not aware of the existence of 
such resources on the Internet. One example is the use of visualisation and animation tools that are great 
aids to program understanding. Although the existence and effectiveness of these tools have been well 
known to computer science researchers in the field, our experience is that few secondary school teachers are 
aware of this. 1 Secondly, teachers might not know where these resources are, even if they are aware of their 
existence. Blind searches on the Internet are likely to be inefficient and sometimes not productive, in terms 
of the time taken to retrieve useful materials. Thirdly, the use of some resources requires a level of technical 
competence that a typical secondary school teacher might lack. Finally, some software tools have to be 
adapted to suit the needs of individual teachers, and without any support or assistance, such tasks could be 
daunting. 

In this paper, we report our experience in the design and delivery of a short course which aims at refreshing 
practising secondary school computer teachers with updated knowledge on teaching and learning with 
computers. We describe how we achieve our goals of providing practical assistance to computer teachers by 
empowering them to effectively exploit Internet resources. Our approach is enabling in that it fosters 
participants' self and lifelong learning beyond the contents c f the present course. We believe that our 
approach is actually applicable to a broader context than ours and therefore would be of interest not only to 
secondary school computer teachers, but also to teacher educators and teachers of other disciplines at all 
levels. 

The rest of this paper is structured as follows. Section 2 introduces the context and goals of our short course. 
Section 3 provides the background of the subject area: computer programming and visualisation tools. 
Section 4 describes how we exploit Internet resources for use in the course. Section 5 describes the 
implementation of the course and the feedback from participants. Section 6 discusses our approach. Section 
7 concludes this paper. 



2 The Teachers Update Course 

2.1 Background and objectives 

Our university has been organising the Teachers Update Course (TUC) annually as a service to local 
secondary schools. It aims at refreshing practising school teachers with updated knowledge on the subject 
areas they teach, and offering advice and assistance on the teaching and learning of the subjects. It serves to 
show our university’ s concerns to secondary education, to share our professional expertise, and to promote 
communication and cooperation between our university and secondary schools. 

TUC consists of a series of half-day short courses that encompass many subject areas such as Use-of English, 
Mathematics, Computer Studies, Physics, and others. This paper reports our experience in th^esign and 
delivery of the course on Computer Studies. Participants of the course were mainly secondary school 
teachers of computer subjects such as Computer Studies and Computer Literacy. 

2.2 The local secondary school context 

In Hong Kong, school teachers are often heavily loaded with both teaching and non-teaching commitments. 
Typically, a teacher has to conduct six to seven lessons per day, each lesson lasting for 35-40 minutes. In 

1 One author of this paper previously taught a class of student teachers in a Postgraduate Certificate in Education 
programme who were major in Computer Studies, and none of them were aware of the existence of program 
visualisation and algorithm animation tools. Similarly, none of the practising computer teachers who participated in the 
Teachers Update Course described in this paper were aware of such tools. 
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addition to such work as lesson preparation, setting and marking tests and examinations, most teachers have 
to share school administrative work as well as leading students to participate in extra-curricular activities. 

In recent years, the Government of the Hong Kong Special Administrative Region (HKSAR) has undertaken 
numerous initiatives to promote the integration of IT into the school curriculum [3]. Since teachers of 
computer subjects are usually more acquainted with the use of computers than other colleagues, they are 
often busily involved in the setting up and management of the IT infrastructure of their schools, and they are 
generally expected to assist other teachers in solving various problems in using IT. 

Increasingly, there are pressures for teachers of all subjects to apply IT in their teaching activities. Many 
teachers have to spend a great deal of time after school hours to attend in-service IT training courses [8,9]. 
However, one common problem they encounter is the limited availability of appropriate educational 
software, and few of them have the time and expertise to develop their own courseware. Moreover, budgets 
are limited in schools for the purchase or development of courseware. 

23 Goals and strategy 

During the planning and preparation of the short course on Computer Studies, the following goals were 
formulated in an effort to maximise the usefulness of the course to the participants: 

• The course had to provide materials that are directly relevant to teaching in schools. 

The course in the previous year was intended to broaden the computer knowledge of school teachers by 
providing updated information on multimedia and their applications. As such, the course was organised 
in the form of a condensed lecture of part of an undergraduate subject, supplemented by demonstrations 
of the applied research work of our staff in the area. Although the subject materials were interesting, 
many teachers subsequently indicated a preference of topics that are more directly related to their own 
teaching in schools. Simply acquiring further knowledge in the computing field was not as welcome as 
knowing something directly useful for solving the problems they encountered in their teaching. 

• The course had to offer practical assistance to teachers. 

Considering the heavy workload of secondary school teachers, any teaching resources must be easy to 
use and demonstrably useful, or they would not be used at all. In selecting the course materials, 
preferences were given to those that are easily and practically applicable in the secondary school 
context. This strategy is also in response to the feedback by teachers in the previous year of their desire 
to learn something that is “more relevant [to their teaching]”. 

• The course should motivate teachers ' interests and empower them to pursue further via self-learning. 

The course was a short one and naturally limited in the amount of teaching materials we could possibly 
provide. Even with a much longer duration, it would still be impossible to inform the teachers 
everything they had to know about the topic. Moreover, even for the same topic, there are considerable 
variations in their needs (for example, due to different teaching styles or their students’ background). 
The same technique useful to one teacher might not work for another. What is more important is to 
foster their ability to pursue the topics further beyond what we offer, whenever they have the need to do 
so. Therefore, from the outset the course was designed to “have an empowering or enabling effect on 
the participants” [9]. We hoped that the course could enable school teachers to acquire what they need 
via self and lifelong learning. jjf* 

Setting the right goals was important, but the real challenge was how to achieve these goals within a few 
hours of contact with the participants. We now outline our strategy as follows. Firstly, we selected a topic 
that would likely interest most computer teachers: computer programming and algorithms. This topic is 
clearly directly related to their teaching. Secondly, we collected useful information and software tools for 
the enhancement of teaching and learning of this topic. Most of these resources were originated from 
overseas and would be hard to access were they not put on the Internet. Thirdly, among them, we selected 
only those information and software tools that were judged to be practically useful in the local secondary 
school context. Finally, we demonstrated to teachers how they could have found and utilised these resources 
on their own through the Internet. 

In retrospect, we believe that although the first step (topic selection) is important in ensuring the relevance 
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of the course, it is our approach in the remaining steps (use of the Internet resources) that would have more 
profound influence to the participants. Our approach will be discussed in detail in Section 6. Meanwhile, we 
briefly introduce the subject area in Section 3 and then elaborate on what we did in the course in Sections 4 
and 5. 



3 Computer programming and visualisation tools 

3.1 Computer programming as a common major part of many computing curricula 

Computer programming and algorithms is usually considered a significant and fundamental component in 
undergraduate computer science education [6]. In most universities, introductory programming and the 
design of elementary algorithms are the first courses that a computing major undergraduate student has to 
take (unless these courses were exempted due to credit transfer or advanced standing). Elementary 
programming courses are also frequently offered as electives to non-computing students with a broad variety 
of backgrounds [10]. 

At the secondary school level, computer programming is historically the major component of a typical 
computer subject. Although the emphasis of learning programming has now been reduced as compared to 
the past, there is, arguably, still a place for it to be included in the secondary school curriculum. In Hong 
Kong, both the Computer Literacy subject (offered to almost all junior secondary students) and the 
Computer Studies subjects (offered as electives to senior secondary students) include programming as a 
major part of the curriculum [2]. 

3.2 Difficulties of teaching and learning computer programming and algorithms 

The teaching of computer programming and algorithms presents a great challenge to educators at both the 
secondary level and the tertiary level [15]. To understand a computer program or an algorithm, the student 
needs to have a good understanding of the internal execution model of computers, as well as the dynamics of 
variables, data structures and control flows in the algorithm [7]. Such concepts are abstract in nature and 
could be difficult to even novice programmers [16], let alone non-computing major undergraduates and 
secondary school students. Indeed, according to our survey to secondary school teacher participants of our 
short course, about 82% of the respondents agreed that computer programming and algorithms are the 
hardest topics to teach. 

There is usually considerable overlap between the contents of a computer subject in a secondary school and 
those of a first year course on computer programming in a university. As such, the difficulties encountered 
by secondary school teachers are in many ways similar to those faced by the professors in universities, as far 
as the teaching of basic computer programming and elementary algorithms is concerned. 

Nevertheless, usually only the academically more capable students will enter universities. As a whole, the 
secondary school student population is less mature in intellectual development and more diverse in their 
academic ability. Compared with university students, many of the secondary school students tend to be less 
motivated and less capable of independent learning; they normally require more guidance in their studies. 

Secondary school teachers are generally less well informed and possess far less resource under their disposal 
than university educators. To our knowledge, a great deal of research has been done in many universities to 
address the difficulties in learning computer programming and algorithms [ 1 ,5,6,7, 1 2fl 5]. Unlike 
universities, however, secondary schools seldom have the resources and expertise to perform similar work to 
solve their problems. In fact, they might not be aware of such research activities. Our approach in the 
course is to facilitate the use of university resources on the Internet by secondary school teachers to solve 
their own problems. 

33 Program visualisation and algorithm animation 

Program visualisation refers to the use of graphical artifacts to represent both the static and dynamic aspects 
of a program [1 1]. Algorithm animation portrays the dynamics of the execution of an algorithm by means of 
animation tools [7]. Educators and researchers have long believed that visualisation and animation are useful 
in helping students understand the abstract concepts and dynamics involved in computer programming and 



61 



algorithms [15]. It is believed that visualisation and animation tools help the learners by displaying in 
concrete form the mental model of the execution of computer programs. Indeed, many universities 
worldwide have been actively researching and experimenting with the use of visualisation and animation 
tools. As a result, a variety of such tools have been developed for different purposes [1,5,6,7,12,15]. Many 
experimental results have been reported that favour the use of such tools for enhancing program 
understanding [6,7,15]. 



4 Exploiting Internet resources for useful educational software tools 

Despite years of active research, program visualisation and animation tools are still not widely used in 
secondary schools , and few such tools designed for teaching and learning are available commercially.2 As 
discussed in Section 2.2, it is often impractical for secondary schools to develop their own tools. 

As computer science researchers and educators, we are aware that many program visualisation and algorithm 
animation tools have been developed as results of research work in various universities. Even though some 
tools have been developed mainly for demonstrating the research ideas and therefore might not have as 
many features as commercial software, most have been designed for teaching and learning. More 
importantly, they are usually available for free and easy access through the Internet for educational purposes. 
To our judgment, there are great potentials of utilising such tools in enhancing teaching and learning in 
secondary schools. 

The idea of utilising research tools on the Internet for enhancing secondary school education is obviously 
appealing and has many advantages over acquiring similar tools by other means. We shall discuss these 
further in Section 6. However, before being convinced of the practicality of this idea, we had two concerns. 
Firstly, although these tools had been successfully applied in the tertiary education context, would they be 
useful in secondary schools as well? Secondly, would secondary school teachers be competent enough to 
make use of these tools that have originally been designed for use by tertiary educators who are technically 
more proficient? 

To develop this idea further, we set out to evaluate the practicality of using Internet resources as teaching 
and learning aids in secondary schools. As program visualisation and algorithm animation do not fall into 
our own research areas, we started our search from only the scarce information that we had. Beginning with 
the Web sites of two well known researchers in these areas that we incidentally came across and made note 
of a few years ago, we followed links over links, and so on. It turned out that there was little difficulty in the 
search of relevant Internet resources. The more tedious and time-consuming task was to evaluate the 
contents of these resources one by one. Even so, within a few weeks' time, we were amazed to have 
collected and evaluated almost a hundred sites of related interest! These resources range from the 
innovative use of common spreadsheet software by researchers in the University of Helsinki [12], to 
ambitious laboratory projects such as the DYNALAB project of Montana State University [1], and 
university students' research projects such as Jeliot [5], 

We selected and evaluated the resources according to several criteria: (1) relevance in content and level to 
the syllabus of secondary school computer subjects, (2) accessibility, (3) flexibility (customisability), (4) 
software and hardware requirements, (5) difficulty in technical content, (6) ease of setup and customisation. 
After evaluation, we decided to recommend about 30 web sites. The contents of these web sites range from 
ready-made animations of common algorithms, to downloadable program visualisation tools that support 
both forward and backward execution [1], and even online animation of user-defined algorithms using 
customisable ‘ actors' in a ‘ theatre-like environment' [5]. 

Through the process of selection and evaluation, we are increasingly convinced of the practicality of our 
approach. Many of the tools we found could be effectively used by people with some elementary knowledge 
of computer programming and concepts of program visualisation. Our participants were computer teachers 
who clearly possess knowledge of the former but not necessarily the latter. Therefore, part of our short 
course was to explain the program visualisation concepts and how they could be useful to aid program 



2 Although most commercial program development environments do provide some limited facilities such as the display 
of the contents of variables during program execution, these are primarily designed to aid software development 
(particularly to aid debugging) by programmers. These facilities are not targeted to beginner learners and usually not 
well suited for the purpose of teaching and learning. 
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understanding. 

5 Course implementation and feedback 

Our course began with discussions on the common problems in developing CAI software. Then we 
introduced various sources from which useful CAI software could be obtained freely or at nominal costs for 
topics in computer subjects in general. These sources included higher educational institutions, students 
pursuing higher education, professional educational bodies, textbook publishers and others. The use of these 
Internet resources was more straightforward and requires no further elaboration other than the provision of 
pointers. 

Next, we introduced the concept of utilising program visualisation techniques for the enhancement of 
teaching and learning, and the corresponding selected Internet resources. For ready made animation tools 
that were straightforward to use, we simply provided pointers and made two representative demonstrations, 
leaving the participants to try and pursue the tools at their own pace after the course. 

A few selected tools, however, were introduced in mich more detail. These tools have one or more of the 
following characteristics: (I ) they were technically more advanced; (2) they could be used in several ways to 
suit different educational purposes; (3) they had features that were particularly useful or illuminating; (4) 
their designs were based on notions that were innovative and less obvious to understand but practically very 
useful. Fortunately, the participants were mainly computer teachers whom could be safely assumed to 
possess the necessary programming skills and concepts to perform the required customisations. Were we to 
simply show the links of these resources, it could be difficult for them to tap the potential benefits of these 
tools effectively. 

The participants were so interested in the selected Internet resources that the course was substantially 
overrun. At the end of the course, participants were requested to complete a questionnaire about their 
background (for planning of future courses) and about how well they felt the course had been organised (for 
evaluation of the present course). Some of the statistics obtained are as follows: 

1. About 82% of the respondents agreed that computer programming and algorithms are the hardest topics 
to teach. 

2. About 90% of the respondents agreed (with 26% strongly agreed) to the statement that “I will try to 
make use of the course materials at school when appropriate”. None disagreed; the rest were 
undecided. 

3. About 87% of the respondents agreed that the course was useful to them; none disagreed and the rest 
were neutral. The same number of respondents agreed that they were satisfied with the course. Some 
felt that the course could have been improved by extending the duration to allow more time for further 
discussions. 

4. All respondents agreed that the demonstration of the Internet resources for teaching was the most 
useful part in the course. 



6 Summary and discussions 

6.1 Characteristics of our approach 

We began with the ideas that program visualisation tools are useful for learning computer programming, but 
such tools are not widely known, of limited availability and hard to develop by secondary scnool teaphers 
themselves. Yet Internet resources abound that could be effectively exploited for use in secondary schools. 
As researchers in the university, by nature of our work we are usually better informed with the availability of 
such resources and the advancement of the latest technologies. In planning and designing the short update 
course for teachers, we positioned ourselves as mentors in the search of relevant teaching resources. We 
aimed at offering practical assistance to secondary school teachers by providing the source of relevant 
information on the Internet, by demonstrating the potential benefits of utilising such information, and by 
guiding them through the solutions to the technical problems that might arise in utilising such information. 
We attempted to motivate the interests of participants, to help them overcome the initial barriers (that is, to 
make “jump start”) so that they could eventually help themselves exploit the vast potentials of Internet 
resources via self and lifelong learning. Incidentally, in so doing, we have exemplified our course as an 
alternative model of “teaching in the information age” in which teachers serve more like a mentor than an 
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authoritative knowledge provider. 



Our approach is characterised in several ways which distinguish it from that of a traditional teacher 
education course. Firstly, our goal was modest yet pragmatic in trying to address a specific but real problem 
that a typical secondary school computer teacher encounters daily: the difficulties of teaching computer 
programming. Secondly, we demonstrated to the participants how Internet resources could be effectively 
and practically utilised for addressing their problems. What is even more distinctive is the recommended use 
of tools developed by researchers with the latest software technologies of the field for use in tertiary 
education. We have argued that both tertiary educators and secondary school teachers share many common 
problems that call for similar solutions. Secondary school teachers could learn a great deal from the 
experience of educators in universities when dealing with their common problems. Finally, the course was 
designed to be enabling and empowering, with the explicit a priori goal that participants could pursue the 
subject further via self and lifelong learning. 

6.2 Reflections and discussions 

On completion of the course with encouraging feedback from the participants, we reflect on the factors 
contributing to our success. We note that a key factor is our decision to take advantages of the use of 
selected Internet resources, especially those from universities worldwide. Firstly, these resources are easily 
accessible to teachers and students alike, as long as they are connected to the Internet. The ease of access 
also minimises the problems that might occur in the distribution and installation of custom developed or 
commercial software. Moreover, the use of educational tools on the Internet is cost-effective. Many of these 
tools have been demonstrated to be effective through their use in universities. They are typically designed 
by computer scientists for demonstrating the advantages of applying their research ideas in education, and 
have subsequently been experimented and evaluated for continuous enhancements, with such evaluations 
adequately documented in their research papers. More importantly, they are available freely or at affordable 
costs. Cost is often a critical factor determining whether an educational software tool will be widely used in 
secondary schools, as resources at their disposal are usually fairly limited. 

Some of the software tools we recommended were developed as prototypes with source codes publicly 
available [12]. They are usually based on sound theoretical principles and accompanied by technical or 
educational papers describing the theory and implementation in detail. Teachers may customise these tools 
to suit their specific needs that might vary due to differences in teaching styles, objectives, and students' 
backgrounds. They may choose to use the whole or part of the tool, or write small program components to 
be integrated with these tools. For computer teachers who are acquainted with and probably interested in 
writing programs, such “lightweight customisation" is usually easier and more feasible than building a 
complete CAI system from scratch. Customisation by users is not normally adequately supported by 
commercial software that comes with no source code and only limited documentation such as operational 
guides.. 

Technologies and knowledge have been advancing very rapidly. On the Internet, new resources keep 
emerging as results of continuous research by academics who explore the latest technologies 6r the 
enhancement of teaching and learning. An example is the experimentation of using 3D visualisation, 
multimedia and virtual reality technologies in education as they emerge [4]. Teachers who are well 
informed of such activities through self-learning on the Internet will be in a better position to make use of 
the latest research results and technologies for continuous improvements to their teaching and learning in 
ways that are not otherwise possible. 

P* 

The use of research tools for teaching and learning is not without problems. However, most of these 
problems would not be deterrent; they could be solved or avoided. Other problems are present in the use of 
other sources of educational software anyway. For instance, research tools are often imperfect, with some 
functionality not fully implemented; but as long as the implemented features are considered useful, the tools 
can be used in part rather than in full. There might be a lack of instant technical support, but many 
researchers who develop the prototypes are keen to collect feedback, as these might be crucial for their 
continuous research work. Inevitably, frequent revisions might occur to these tools for research purposes, 
but if the teacher finds an earlier version useful, that version could be downloaded and kept for use instead 
of relying on its availability at the source. 




7 Conclusions 



University educators possess the necessary resources, expertise and freedom to fulfil their roles of 
performing experimentation and researches, and producing prototypes to demonstrate the usefulness of their 
innovative ideas. In comparison, secondary school teachers are too occupied with teaching activities and 
other professional commitments. Most teachers cannot afford the purchase of expensive commercial 
software for teaching, nor do they generally have the capacity of developing appropriate educational 
software on their own. Success of integrating IT in the school curriculum is critically determined by the 
availability of easy-to-use and adaptable tools that satisfy the diverse needs of teachers and students of a 
variety of backgrounds in different contexts. 

The Internet has provided a medium on which tertiary educators can make their resources and experience 
publicly available to be shared by all, including secondary school teachers. Around the world, numerous 
tertiary educators have gladly done so as part of their service to the community. Unfortunately, such 
resources are largely under-utilised by secondary school teachers, due to reasons such as the lack of 
knowledge and technical competence. For computer teachers, these barriers are relatively easy to overcome, 
as long as appropriate support and assistance is provided. For teachers of other disciplines, more help might 
be required. Ultimately, secondary school teachers have to learn, adapt and use these resources by 
themselves, and to keep themselves updated via self and lifelong learning to respond to the rapid changes 
that the world has been undergoing. 

In this paper, we have reported our experience in the design and delivery of a short course that has 
progressed towards this direction. Our course also exemplifies itself as one possible model of “teaching” as 
“facilitating the self and lifelong learning of the participants”. Most tertiary educators have now become 
regular users of Internet resources for enhancing their teaching and learning. It should not be long before 
secondary school teachers have to follow suit. What we have contributed is but a small part of the continuing 
collaborative effort to empower teachers to use IT effectively in secondary schools, and ultimately to better 
education of our younger generations. 
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Student teaching is an important part of teacher training programs. With the 
emerging and the widespread use of the Internet, it is important to consider how 
this crucial stage of teacher development can be facilitated by the use of the 
technology. In order to create for a user-oriented and research-based web 
environment, this project was designed to explore problems that student teachers 
experience. During the internship year, student teachers filled out a self-report 
critical problem questionnaire five times in two periods, one in each semester. 

The student teachers were asked to pick one critical problem that they had tried 
hardest to solve in the day or the week. In the survey, they wrote down the ways 
to solve the problem as well as the resources they used in the process. The 
results showed that peer student teachers were those whose help were mostly 
sought. Over 90% of the means to solve the problem was face-to-face. It is 
summarized that student teachers may need three types of proximity for 
problems: Professional, emotional and physical. To provide rich interpretation to 
the problems, it is suggested that an experience database with focused case study 
discussion forum may be of help to student teachers' problems. 

Keywords: Student teacher, Student attitude, Teaching experience, Internet 
use 



1 Introduction 

Student teachers are in the process of becoming a teacher. Fresh from the university, student teachers are 
often full of ideals and enthusiasm. Entering the real world of teaching, however, they are likely to 
experience problems and difficulties that can be termed “reality shocks” (Wubbels, Creton, Hooymayers & 
Hoi vast, 1982). With the Internet technology becoming more accessible and versatile, there are an increasing 
number of web-based projects to assist student teachers (Georgi & Crowe, 1998). Instead of building the 
technology first and assessing the student teachers later, this project intends to design a research-based and 
student teacher-oriented web environment. 

This study proposes to examine the needs of the student teachers and use the results as basis tq^onstruct a 
web environment. During a one-year internship, a class of 76 student teachers were asked to participate in 
the survey and interviews for their problems and difficulties, as well as the resources they used to resolve 
their problems. The analysis of the problems will be used to develop the guidelines and the structure of the 
website. 



2 Theoretical Background 

2.1 The problems of the student teachers 

Numerous studies have been done to understand the problems and "reality shocks" that student teachers 
encountered. In an extensive review, Veenman's (1984) collected 91 research studies in the last two decades. 
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His summary of the findings suggested eight categories of problems, including managing student, 
motivating students, dealing with individual differences, evaluating students’ work, communicating with 
parents, organizing class work, obtaining supply and teaching material, and tackling individual student's 
problems. Chen & Chen (1999) critiqued the previous researcher-designed surveys and used student 
teacher’s journals as a means to understand their problems. They collected 800 student teachers’ journals and 
used Multidimensional Scaling to analyze the data. The major categories of student teacher's reality shock 
included status uncertainly, students' attitudes and disciplines, conflicts between the decision maker and the 
doer, the negative-reinforcement style of management, the working ethics of teacher and staff, as well as the 
relationships among school members (Chen & Chen, 1999). 

While many studies addressed the problems that student teachers encounter, most of them focused on why 
the problems occurred and how to solve the problems for them. Very few, on the contrary, investigated how 
student teachers solved their problems. Questions regarding whom student teachers asked for help and what 
resources they used in solving their problems were seldom discussed. The purpose of study, therefore, is not 
to postulate another possible cause of the problems, rather, is to find out what resources student teachers use 
to solve their problems, and how technology can help expand this access. 

2.2 The problems with the technology 

The use of Internet technology for teacher training has received growing attention. E-mail is perhaps still the 
most widely used means to encourage communication between the supervising teacher and student teachers 
(Nabors, 1999). More recent developments include more sophisticated design such as electronic portfolio to 
promote reflection and performance-based assessment (Georgi & Crowe, 1998). Morley's (1999) project 
uses WebCT, an Internet-based interface, for course syllabus, class notes, hyperlinks, as well as bulletin 
boards for faculty and students in pre-service method courses. The National Science Council in Taiwan in 
recent years has funded several projects in building web-supported environment student teachers in areas 
such biology, math, science and technology (Guo, 1999). 

When new technology is added to student teaching, however, some precautions are warranted. As an add-on, 
the help it provides may not be critical to the user's needs nor adopted by the user in a long run. Examples 
can be observed in many websites where only few messages are found in the discussion area. As Hsu & 
Bruce (1998) observed, teachers in distance education often fail to communicate with their distance students 
because their pedagogical strategy with the new technology does not supply the necessary cues that is 
acquainted by the students in their face-to-face environment. Therefore in this project we want to explore 
student teachers’ current situation before designing the website. 



3 Methods 

A total of 35 student teachers from 11 subject areas of junior and high schools participated in this one-year 
study. The participants were all recent graduates from university or graduate schools of the same university. 
To sample the student teacher's experiences with problems and difficulties across the internship year, the 
critical problem survey involved two rounds of sampling periods, once in the end of the fall semester and 
once at the end of the spring semester. 

During the first semester, student teachers were asked to fill out a questionnaire once a week for five weeks. 
Every week they had to pick one most critical problem in the past week. Three open-ended questions were 
designed to elicit the most critical problem that demanded the most of the student teachers' tinte and energy 
to solve. The three open-ended questions were: 1) What is the most critical problem you have experienced 
during the week? 2) How do you resolve the problem? And what resources do you use? 3) At the end of the 
week, was the problem resolved? If not, how would you like it to be solved? 

In addition to the open-ended questions, there was a chart where student teachers had to check boxes for the 
people they had talked to regarding to the problems they were trying to resolve. The choices included the 
cooperating teacher, the supervising teacher, the student teachers in the same subject area and different 
subject area from the same university, the student teachers in the same school but from different university, 
the family, the roommate, none, and others. They were also asked how many times they have made the 
contact and by what means the communication was made. The choices included face-to-face, phone, e-mail, 
and others. 
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The questionnaires were first mailed out to the student teachers. After the initial data collection, it was found 
that the returned rate was too low. Therefore, additional short telephone interviews with 25 students were 
arranged. The interview also provided a little more in-depth background for their problems and difficulties. 
At the end of the spring semester, the same questionnaire was filled out daily for five days with the help of 
telephone interviews. Regular attendance to the student teacher’s monthly meeting and small group 
discussions also informed the interpretation of the data collected. 



4 Results 

4.1 Student teacher’s problems 

The results of the self-reported questionnaire and the transcript of the interview were coded by two 
researchers and two research assistants. The coding scheme originally used was Chen & Chen’s (1999) 
findings of six categories, but the emerging themes of the data yields to the following four major categories 
in student teachers' problems. 1) Ambiguity of the status, including conflicts with the cooperating teachers 
for competing authority in the class; conflicts with school administrators in terms of task assignment; and 
conflicts with the school culture in terms of the feeling of unfit to the school physical environment, goals, 
and life styles. 2) Lack of professional knowledge, including subject knowledge, teaching skills, class 
management skills, and skills for student discipline problems. 3) Relationship with cooperating teachers, 
administrators, and students; including problems in making their needs known; and in dealing with small 
groups and gender issues. 4) Confusion in teaching as career goals, including conflicts between the ideal 
and reality. 

4.2 Ways to solve the problems 

When stressed by a problem, student teachers did not always know how to solve it. They usually consulted 
people for solutions. Categories of people whose help were sought after were coded from both the 
questionnaire and the interview. 1) Cooperating teachers, to ask for assistance or professional suggestions on 
classroom management and teaching skills. 2) Other teachers of the same subject area, for content 
knowledge and student discipline problems. 3) Other student teachers, to seek answers and condolence from 
others about conflict with the cooperating teacher and students’ disciplines; also for relationship and cultural 
adjustment. 4) Solving the problem by oneself, such as trying out ones' own new ideas, making more effort 
to learn new things, adjusting attitude, accepting the reality, or simply enduring it. 

Depending on the nature of the problem, other resources were sought for specific information. For legal 
issues, for example, some student teachers sought help from higher up authorities. In terms of technology, a 
few student teachers used the Internet to find teaching material and lesson plans. Not every problem had a 
solution, however. During our talk during the interview and in informal settings, quite a few students 
indicated that they often choose to passively accept the situation or to give up thinking for solutions. The 
following figure is a summary of the results from the questionnaire about the help the student teachers 
sought (see Figure 1). The results showed that about 47% of talks were with the other student teachers, 
where 27% were from the student teachers in the same school. About 22% of help was received from the 
cooperating teacher, and another 17 % were from family and roommate. Only 1% was from their supervising 
teachers. Among all the communication means, 92% were face-to-face, 6 % were by phone, and 2% were by 
e-mail. 
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Figure 1 People whose help student teachers sought from 



Help Sought (Frequency, Total=281) 




■ 1 Cooperating teachers 


12%-, 


EJ 2 Supervising teachers 




■ 3 ST with the same subject area 




□ 4 ST with different subject area 


1 % 




■ 5 Other ST in the same school 


1 3 % 


B 6 Family 




□ 7 Roommate 


27 % 


□ 8 None 




H 9 Others 







5 Discussion 

According to the analysis, the problems of student teachers ranged from personal to professional. The 
solutions, although ranged from professional guidance to personal camaraderie, are limited to face-to-face 
communication. To contemplate what will help student teachers in solving their problems, it may be useful 
to postulate what they needs are. 1) Professional proximity. Being a novice, a student teacher may eager to 
know how others would have done differently. Those who are authoritative in professional fields, such as 
cooperating teachers, are likely to be pursued for instructional and managerial guidances. 2) Emotional 
proximity. Besides professional guidances, student teachers need to find emotional support to feel that they 
were not alone. It is also safer to talk to peers for issues of role adjustment and interpersonal relationships in 
schools. 3) Physical proximity. Those who are physically close ( 66 %) are more readily to help. Therefore, 
when physical gatherings are not available, the help seeking channels can become seriously impeded. 

Based on the above findings, we can begin to think about the design of a web-supported environment. The 
employment of a student teacher website should have features that provide additional or alternative support 
that take the above three types of proximity into account. The complexity of the problems and the limited 
access to solutions suggested that a case method that the user can criss-cross for multiple interpretations may 
be appropriate for learning in a complex knowledge domain (Spiro & Jehng, 1990). The following 
components are proposed in the website to be built. 1) A student-teaching case database. To provide 
experiences of other student teachers in a form of journals, including description of and reflection on various 
aspects of student teaching. This database is both outlined and keyword searchable. Hyperlinks to other 
similar cases can be also built. Student teachers can access to a peer’s life lessons without having to have an 
appointment with him. 2) Guidelines and suggestions. Also included in the database are written guidelines 
and suggestions from academics, experts, experienced teachers and student teachers on the same topics as 
the above case database. Links to other web resources regarding professional information will also be added. 
Student teachers can reach specific information for guidance without much effort. 3) Focuseucase study 
discussion forum. To provide threaded bulletin boards on selected cases from the database. With shield 
identity, student teachers can find emotional support without being exposed. The cases can be rotated on 
weekly bases and among different subject matters. 4) Annotated video components of teaching. Also 
included in the database can be video clips of exemplar teaching of cooperating teachers as well student 
teachers' teaching. Written comments can be added by both the cooperating teachers and supervising 
teachers. This is a good place to engage a productive conversation among the triad of the student teacher, the 
cooperating teacher, and the supervising teacher. 

It is hoped that with the aid of the technological power, the student teachers will have better chances to solve 
their problems and they should feel more empowered in their first full-time exposure to the real world of 
teaching. 
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Western research recognises [1 ] within the CS academic community pressure on 
its curriculum and teaching methodology brought about by the “evolutionary” 
nature of the discipline. This continuing need to avoid obsolescence in 
curriculum, which is produced by changing needs of industry and advances in 
research, is accompanied by other international and more local issue issues. 

Current research identifies several techniques which may be used to motivate 
and support CS learning in such an environment. This paper examines the 
implications of these findings to Asia, and particularly to mainland China, based 
on the personal reflections of this researcher on her own studies in China. 
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1 Areas of concern in Western CS education 

Some modern issues of concerns in CS education [1] include attrition due to poor motivation/leaming 
difficulty at CS1 level, dealing with students from a wide range of backgrounds with different learning styles 
and teaching the problem-solving and lifelong learning skills demanded by industry and research 

2 Physical solutions 

2.1 Providing Motivation Through Active And Participatory Learning 

Active [2] and participatory learning [3] are techniques that are proposed to help motivate learners. Some 
methods which can be used include providing opportunities in “modified lectures” for paired response to 
questions posed by the lecturer or students discussing the notes they have taken during the lecture and 
helping to correct misunderstandings. Others have used role-play to demonstrate structures and protocols eg. 
arrays, linked lists or token passing protocols. 

22 Learning programming through pattern recognition 

In dealing with the learning of programming at the basic level, many researchers have examined the issues 
of teaching CS1. There is much interest in the use of patterns in aiding students’ comprehenskip of basic 
programming and the integration of this knowledge [4 Clancy & Linn]. 

Clancy and Linn comment on the fact that design patterns are of great importance in software engineering 
and 00 design and that to, to some extent, programming knowledge consists partially, at the cognitive level, 
as patterns (or schemas). However they show that, while the use of patterns is helpful in integrating 
knowledge, new programmers do not naturally infer patterns and sometimes find it difficult to understand 
“expert patterns”. Clancy and Linn [4] and Johansson [5] advocate the use of a wide-range of contextualised 
examples and case studies to support the teaching of basic programming skills. 

2.3 Adapting pedagogical styles to deal with social, cultural and gender issues 

Research shows that students from different cultures or of different genders display different attitudes to 
computers and learning. In a two-year study of female and international CS students at Carnegie Mellon 
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University [6] issues which arose included the perception by some female students that the ‘ purpose* of 
computing needed to be defined within introductory CS courses. While they displayed a high-level of 
interest in the computing process they needed to be able to contextualise this process “within a larger 
purpose”. They also displayed a lower level of “attachment” to their computers than did male students on 
the same course and expressed some relief as they discovered that CS education covered a wide-range of 
topics 

Other research [7] [8] points out that there is a link between culture and learning style. Assertions made in 
this research indicate that Chinese students (studying overseas) would find it easier to understand and apply 
theoretical principles within programming than would a similar group of Western students. In their study, 
Fisher, Margolis and Miller [6] discovered that international female students on their course showed the least 
“attachment” to computers or computing and used pragmatic reasoning (such as employability) for their 
choice of major. 

The conclusion here is that some allowance has to be made for cultural and gender preferences within the 
teaching of CS. While it is possible to provide an inclusive focus within lectures, there is, however, some 
may be a more pressing need to be able to adapt tutorial material for different styles and preferences. 

2.4 Problem-solving for lifelong learning 

It has been noted [9] that many students who have difficulties across the first year of CS as a whole do not 
know where to start with a task, regardless of the subject area. 

Some effort has been made to incorporate training in problem solving skills and techniques in to early CS 
education to deal with this problem . This ranges from the use of Edward de Bono* s tools for lateral 
thinking to the development of Polya* s approach of Understand, Design and Review [9] for problem solving 
and offering courses in these techniques within, or parallel to, early programming subjects 

2.5 Web-Mediated solutions 

With the problems imposed by large classes, and the large range of individual approaches needed to deal 
with some of student learning issues raised above, CS academics have been some of the first to develop and 
use web-mediated learning environments for enhancing student learning in CS. 

As I have pointed out ([8], [9]) the Web provides a vehicle for the development of the learning environment 
and teaching can be structured to develop lifelong learning skills and to cater for the expectations and 
learning styles of students from different cultures and backgrounds. 

Early Australian examples of this style of teaching in CS education are many. Recent Australian examples of 
the use of the WWW in CS education abound. Boalch [10] provides an examination of the use of the WWW 
as a support medium for the delivery of a first year unit in Information Systems at Curtin University. He 
provides an evaluation of site utilisation and user feedback in the case where subject information and course 
details were provided on the WWW for students. 

The Eklunds [11] examine the use of the WWW to supplement traditional IT teaching. They provide case 
studies of two examples of the re-structuring of traditional forms of IT course for Web-delivery. Jones [12] 
of Central Queensland University gives details of case study involving the design, presentation and 
evaluation of an undergraduate unit in Systems Administration taught completely via the W^/W to on- 
campus and distance students. 

3 Reflection on CS Education in Mainland China 

The following two stories are taken from some interpretive tales which I wrote after two separate periods of 
studying and teaching in China. They draw a picture of the role of the computer on campus in Nanjing 1 995) 
and in Jinan (1998). 

A Visit To The Computer Centre 1995 

I managed to pay a visit to the University Computer department (I was a Computer lecturer myself at 
the time in Australia). This was a definite culture shock. The computers, 386s and old at the time, were 
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kept in a special air-conditioned and carpeted room. People wore white coats and slippers if they wanted to 
use them. Most students (and only the best study computers) were doing basic Basic programming. I tried 
to investigate whether they used Windows, or anything modem, but the lecturer was only interested in the 
length of computer courses in Australia. There seemed to me to be no parallels in our courses at all. The 
students seemed only to learn Basic programming {I wondered what job this would qualify them for!] . It 
seemed to that things like word processing [the Chinese have a special keyboard and it takes 5 keys together 
to create one character] were a matter for female secretaries and did not enter the arena of the university. I 
tried to explain the issue of the ‘computer as a tool* but I could see that the body language was saying 
‘ Crazy Westerner!’ when I tried to put across the concept of teaching less -able, or even all students, to use 
computers. Computers are for the young and highly intelligent in China. 

A Visit to the Internet Centre 1998 

It was surprised to find the computer was still as remote as ever from the everyday life of the average 
student. Computers, 486s by now, still lived in splendid isolation in carpeted rooms, and students still wore 
special slippers to use them. Still no Windows and still basic Basic. 

I had imagined that the cutting edge of technology would be a little different to that which we had at home. I 
was a little surprised though to find out the process which I had inadvertently become involved with. I 
worked for six weeks with some highly creative young teachers to try and develop an intranet from an old 
CAD classroom (486s with no hard disks), one modem Pentium in a building several hundred metres away, 
one modem and a collection of legal and not-so-legal software. The Internet Centre turned out to be a 
heavily guarded room about the size of an average Western kitchen with a little row of computers along one 
wall, filled with a large collection of discarded technology and useful pieces of wire. 

Major problems for the Chinese academics was their lack of ability in reading English as the ‘install’ 
dialogue boxes sped past on the screen. The problem for me was that I read Chinese much more slowly than 
they could read English. All the online-help in the world did not help us, installation was a slow process! 
We often laughed at the problems because we were all engineers and computer scientists. Not really the 
type of people who are famed for their linguistic abilities, but the monopoly of the Internet by the English 
language is certainly a problem in China. 

I left before the networking was done. I did manage to complete a bilingual virtual library and an English 
home page for the Institute (with the help of some young teachers) and to teach a couple of them to use 
FrontPage. I gave lectures to many of the final year students and their teachers. Certainly no lack of 
enthusiasm here - just a lack of technology and English teachers! 

3.1 Chinese Teaching Practice and Computer Based Education. 

The combination of a Confucian philosophy and commonly accepted teaching models means that, in 
universities and colleges, all subjects are taught lecture -style to large groups. However to a Western none of 
the common CS teaching problems established above is observed in daily teaching and research. 

From a Western perspective motivation remains very high among students as they strive to master modem 
hardware and software. Gender issues and the ability to attract female students do not appear to be a great 
concern and classes appear to display a balance between males and females. Learning problems do not 
appear to be the major difficulty experienced and researched in the West. 

The major problem appears to be curriculum. The Chinese system has been one that has relied <6if a national 
curriculum in all sectors of education and changes in the software and hardware used and taught havt‘ not 
been allowed. During April 1998 (China Daily, 1998) the Ministry of Education announced major 
adjustments in the University system with corresponding changes to the High School curriculum and 
schoolbooks, which provide some hope that this issue will be addressed. 

A national curriculum which has not kept pace with changes computing practice in Chinese industry and 
commerce, and even the home, has caused a demand for Western computer manuals in Chinese translation 
and the increase in number of private providers offering training in modem computer applications and the 
Internet. Many young teachers and their students are becoming competent users of modern software (eg 
Windows 98/NT, Office97, object-oriented software) which is not available within the Higher Educational 
system by turning to these private providers. This leads to disaffection and difficulties for both teachers and 
their students. 




3.2 The Future 



As well as the obvious improvements to connections, access speeds and call charges which are currently 
being made by CERNET, wider issues to be faced are the development of Chinese language software and 
WWW pages to improve the take-up of the Internet in China as a whole. This is being carried out in an 
environment of large-scale educational reform which will need to take into account the effect of the Internet 
on accepted Chinese teaching practice and pedagogy. 

4 Conclusions 

It is hard to imagine that, even within the next ten years, the Chinese economy might begin to develop and 
maintain a systemic hardware and software infrastructure within higher education. While it is easy to 
envisage the limited availability of the Internet for research students, and especially in the nationally funded 
universities and those around Beijing, the provincial lecturer has the doubly difficult task of persuading the 
older and therefore more powerful academics to accept new technology and to make drastic changes to their 
teaching style to incorporate it. 

I have proposed elsewhere [9] that an effective conceptual framework for the development of an online 
learning environment might be one which is based on expected pedagogical outcomes. Therefore one model 
for China would be to concentrate on the development of online teaching content which would be a resource 
for guided and collective discovery learning (see above). This might begin with the development of Chinese 
language link pages to English language resources such as comprehensive virtual libraries and databases. 

Academic staff development in technology is both very easy and very difficult. Young Chinese academics 
are as adept as their Western counterparts in their understanding and use of cutting-edge technology. Their 
progress is however hampered by their English language skills. This is especially apparent when one is 
made aware of the lower standards of English language required for technical subjects and the datedness (or 
nonexistence)of the technical vocabulary taught at university level. This appears to be one of the most 
pressing problems for the Chinese universities to grapple with and solve. 

CS education research has shown a need for pattern recognition, motivation and problem solving skills as 
aspects of life-long learning. These can be supplied through the medium of web-mediated adaptive tutoring 
which can be used to augment face-to-face teaching but great efforts will need to be made to use these 
effectively within the current Chinese pedagogical framework. 
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Little work to date has addressed the effects that problem/solution 
representations have on collaborative learning processes. This paper outlines 
empirical and theoretical reasons why the expressive constraints imposed by a 
representation and the information that a representation makes salient may have 
important effects on students’ discourse during collaborative learning. It then 
reports initial results from a pilot study. Students worked together in pairs on 
hypertext-based “science challenge” problems. Pairs used either free text, matrix 
or graph representations of evidence, with two groups assigned to each kind of 
representation for a total of six groups. Analysis of discourse transcripts 
suggests that these representations have quite different effects on the extent to 
which students discuss evidential relations. 
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1 Introduction 

Decades of research into cognitive and social aspects of learning have developed a clear picture of the 
importance of learners’ active involvement in the expression, examination, and manipulation of their own 
knowledge, as well as the equal importance of guidance provided by social processes and mentorship. 
Recently these findings have been reflected in software technology for learning: systems are now providing 
learners with the means to construct and manipulate their own solutions while they are being guided by the 
software and interacting with other learners. My work is within this spirit, providing representational tools in 
support of collaborative learning. Representational tools may range from basic office tools such as 
spreadsheets and outliners to “knowledge mapping” software. Such tools help learners see patterns, express 
abstractions in concrete form, and discover new relationships [4, 8], These tools can function as cognitive 
tools that lead learners into certain knowledge-building interactions [3,7]. 

For a number of years, my colleagues and I have been building, testing, and refining a diagrammatic 
environment (“Belvedere”) intended to support secondary school children’s learning of critical inquiry skills 
in the context of science. The diagrams were first designed to capture scientific argumentation, and later 
simplified to focus on evidential relations between data and hypotheses. This change was driven in part by a 
refocus on collaborative learning, which led to a major change in how we viewed the role of the interface 
representations. Rather than viewing the representations as medium of communication or a formal record of 
the argumentation process, we came to view them as resources (stimuli and guides) for conversation [12 4 , 17]. 
Meanwhile, various projects with similar goals (i.e., critical inquiry in a collaborative learning context) were 
using radically different representational systems, such as hypertext/hypermedia [6, 9, 13, 22]; node-link 
graphs representing rhetorical, logical, or evidential relationships between assertions [11, 14, 19, 20] 
containment [1], and evidence or criteria matrices [10]. 

Both empirical and theoretical inquiry suggests that the expressive constraints imposed by a representation 
and the information (or lack thereof) that it makes salient may have important effects on students’ discourse 
during collaborative learning. Specifically, as learner-constructed external representations become part of the 
collaborators’ shared context, the distinctions and relationships made salient by these representations may 
influence their interactions in ways that influence learning outcomes. However, to date little systematic 
research has undertaken to explore possible effects of this variable on collaborative learning, except for [5], 




This paper motivates and describes our research and reports initial results from such a study. 

2 Representational Guidance 



The major hypothesis resulting of this work is that variation in features of representational tools used by 
learners working in small groups can have a significant effect on the learners’ knowledge-building discourse 
and on learning outcomes. The claim is not merely that learners will talk about features of the software tool 
being used. Rather, with proper design of representational tools, this effect will be observable in terms of 
learners’ talk about and use of subject matter concepts and skills. We have begun investigations to determine 
what features have what kind of effect. This section develops an initial theory of how representations guide 
learning interactions, and applies this analysis to make specific predictions concerning the effects of selected 
features of representational tools. The discussion begins with some definitions. 

Representational tools are software 
interfaces in which users construct, 
examine, and manipulate external 
representations of their knowledge. 

Our work is concerned with 
symbolic as opposed to analogical 
representations. A notation/artifact 
distinction [16] is critical to the 
theory, as depicted in Figure 1. A 
representational tool is a software 
implementation of a 

representational notation that 
provides a set of primitive 
elements out of which 
representations can be constructed. 

(For example, in Figure 1, the 
representational notation is the 
collection of primitives for making 
hypothesis and data statements and 
“+” and links, along with rules 
for their use.) The software developer chooses the representational notation and instantiates it as a 
representational tool, while the user of the tool constructs particular representational artifacts in the tool. 
(For example, in Figure 1 the representational artifact is the particular diagram of evidence for competing 
explanations of mass extinctions.) 

Learning interactions include interactions between learners and the representations, between learners and 
other learners, and between learners and mentors such as teachers or pedagogical software agents. Our work 
focuses on interactions between learners and other learners, specifically verbal and gestural interactions 
termed collaborative learning discourse. 

Each given representational notation manifests a particular representational guidance, expressing certain 
aspects of one’s knowledge better than others do. The concept of representational guidance is borrowed from 
artificial intelligence, where it is called representational bias [21]. The phrase guidance is adopted here to 
avoid the negative connotation of bias. The phrase knowledge unit will be used to refer generically to 
components of knowledge one might wish to represent, such as hypotheses, statements of fact; concepts, 
relationships, rules, etc. Representational guidance manifests in two major ways: f**' 

♦ Constraints: limits on expressiveness, i.e., which knowledge units can be expressed [15]. 

♦ Salience: how the representation facilitates processing of certain knowledge units, possibly at the 
expense of others [8]. 

As depicted in Figured, representational guidance originates in the notation, but affects the user through 
both the tool and artifacts constructed in the tool. 

The core idea of the theory may now be stated as follows: Representational tools mediate collaborative 
learning interactions by providing learners with the means to articulate emerging knowledge in a persistent 
medium, inspectable by all participants, where the knowledge then becomes part of the shared context. 
Representational guidance constrains which knowledge can be expressed in the shared context, and makes 
some of that knowledge more salient and hence a likely topic of discussion. The discussion now turns to 
three predictions based on differences between representational notations. 
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2.1 Representational notations bias learners towards particular ontologies 

The first hypothesis claims that important guidance for learning interactions comes from ways in which a 
representational notation limits what can be represented [15, 21]. A representational notation provides a set 
of primitive elements out of which representational artifacts are constructed. These primitive elements 
constitute an ontology of categories and structures for organizing the task domain. Learners will see their 
task in part as one of making acceptable representational artifacts out of these primitives. Thus, they will 
search for possible new instances of the primitive elements, and hence (according to this hypothesis) will be 
guided to think about the task domain in terms of the underlying ontology. 



For example, consider the following interaction in which students were working with a version of Belvedere 
that required all statements to be categorized as either data or claim. Belvedere is an ’’evidence mapping" 
tool developed under the direction of Alan Lesgold and myself while I was at the University of Pittsburgh 
[18, 19, 20], The example is from videotape of students in a 10 th grade science class. 

S 1 : So data, right? This would be data. 

S2: I think so. 

SI: Ora claim. I don’t know if it would be claim or data. 

S2: Claim. They have no real hard evidence. Go ahead, claim. I mean who cares? Who cares what they 
say? Claim. 

The choice forced by the tool led to a peer-coaching interaction on a distinction that was critically important 
for how they subsequently handled the statement. The last comment of S2 shows that the relevant 
epistemological concepts were being discussed, not merely which toolbar icon to press or which 
representational shape to use. 



2.2 Salient knowledge units are elaborated 



This hypothesis states that learners will be more likely to attend to, and hence elaborate on, the knowledge 
units that are perceptually salient in their shared representational workspace than those that are either not 
salient or for which a representational proxy has not been created. The visual presence of the knowledge unit 
in the shared representational context serves as a reminder of its existence and any work that may need to be 
done with it. Also, it is easier to refer to a knowledge unit that has a visual manifestation, so learners will 
find it easier to express their subsequent thoughts about this unit than about those that require complex 
verbal descriptions [2]. These claims apply to any visually shared representations. However, to the extent 
that two representational notations differ in kinds of knowledge units they make salient, these functions of 
reminding and ease of reference will encourage elaboration on different kinds of knowledge units. 
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Figure 2. Example of Elaboration Hypothesis 

For example, consider the three representations of a relationship between four statements shown in Figure 2. 
The relationship is one of evidential support. The middle notation uses an implicit device, containment, to 
represent evidential support, while the right-hand notation uses an explicit device, an arc. It becomes easier 
to perceive and refer to the relationship as an object in its own right as one moves from left to right in the 
figure. Hence the present hypothesis claims that relationships will receive more elaboration in the rightmost 
representational notation. 

The opposite prediction is also plausible. Learners may see their task as one of putting knowledge units “in 
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their place” in the representational environment. For example (according to this competing hypothesis), once 
a datum is placed in the appropriate hypothesis container (Figure 2b) or connected to a hypothesis 
(Figure2c), learners may feel it can be safely ignored as they move on to other units not yet placed or 
connected. Hence they will not elaborate on represented units. This suggests the importance of making 
missing information salient. 



2.3 Salience of missing units guides search 

Some representational notations provide structures for organizing knowledge units, in addition to primitives 
for construction of individual knowledge units. Unfilled “fields” in these organizing structures, if 
perceptually salient, can make missing knowledge units as salient as those that are present. If the 
representational notation provides structures with predetermined fields that need to be filled with knowledge 
units, the present hypothesis predicts that learners will try to fill these fields. 



For example, Figure 3 shows artifacts from three notations that differ in salience of missing evidential 
relationships. In the textual representation, no particular relationships are salient as missing: no particular 
prediction about search for new knowledge units can be made. In the graph representation, the lack of 
connectivity of the volcanic hypothesis to the rest of the graph is salient. Hence this hypothesis predicts that 
learners will discuss its possible relationships to other statements. However, once some connection is made 
to the hypothesis, it will appear connected, so no further relationships will be sought. In the matrix 
representation, all undetermined relationships are salient as empty cells. The present hypothesis predicts that 
learners will be more likely to discuss many relationships between statements when using matrices. 
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Figure 3. Example of Salient Absence Hypothesis 



2.4 Predicted Differences 



Based on the discussion of this section, the following predictions were tested in the study reported below. 
The symbol indicates that the discourse phenomenon at the beginning of the list (concept use, 
elaboration, or search) will occur at a significantly greater rate in the treatment condition(s) on the left of the 
symbol than in those on the right 

Concept Use: ( Graph, Matrix) > (Container, Text, Threaded Discussion). The Graph and Matrix 
representations require that one categorize statements and relations. This will initiate discussion of the 
proper choice, possibly including peer coaching on the underlying concepts. The Container, Text, and 
Threaded Discussion representations provide only implicit categorization. Students may discuj&*placement 
of information, but this talk is less likely to be expressed in terms of the underlying concepts. 

Search for Missing Relations: Matrix > (Container, Graph) > (Text, Threaded Discussion). The matrix 
representation provides an empty field for every undetermined relationship, prompting participants to 
consider all of them. In Graphs or the Container representations, salience of the lack of some relationship 
disappears as soon as a link is drawn to the statement in question or another is placed in its container, 
respectively. Threaded Discussion does not specifically direct searches toward missing relationships. 

The Elaboration hypothesis was not tested independently of the Search hypothesis in this study. 



3 An Initial Study 
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This section reports on an initial study that was conducted to identify trends suggesting that there is a 
phenomenon worthy of further study; and to refine analytic techniques. Specifically, the study examined 
how the amount of talk about evidence and the amount of talk about the epistemological status of 
propositions (empirical versus theoretical) differed across three representational tools, and provided 
qualitative observations to guide further study. 

3.1 Design 

Six pairs (twelve participants) were distributed evenly between three treatment conditions in a simple 
between-subjects design. The three treatment conditions corresponded to three notations: Text, Graph, and 
Matrix. These notations differ on more than one feature, such as ontology, whether inconsistency relations 
are represented, and visual and textual notations. I intentionally chose this research strategy (instead of 
manipulating precisely one feature at a time) in order to maximize the opportunity to explore the large space 
of representations within the time scale on which collaborative technology is being adapted. 

3.2 Method 

3.2.1 Participants 

Middle-school boys were recruited by my assistant (Cynthia Liefeld) from soccer practice. Two pairs of 
participants were run in each of the three conditions. Each pair consisted of boys who knew each other, a 
requirement intended to minimize negotiation of a new interpersonal relationship as a complicating factor. 

3.2.2 Materials 

Software. Three existing software packages were used: Microsoft Word (Text), Microsoft Excel (Matrix), 
and Belvedere (Graph). Groups using MS Word were not prohibited from using its typographical devices 
such as different typefaces, styles, lists, etc. We did not restrict participants' appropriation of typographical 
devices for organizing information, but neither did we encourage any particular use of the textual medium. 
Groups using MS Excel were provided with a prepared matrix that had the labels "Hypotheses" and "Data" 
in the upper left comer, and cells formatted sufficiently large to allow entry of textual summaries of the same. 
Participants were specifically told to enter hypotheses as column headers, data as row headers, and to record 
the relationships in the internal cells. The Graph condition used Belvedere. The version of Belvedere used 
(2.1) provides rounded nodes for hypotheses, rectangles for data, and links for consistency and inconsistency 
relations between them. Hypothesis and data shapes are filled with textual summaries of the corresponding 
claims. 

Science Challenge Problems. Participants were presented with “science challenge problems” in a web- 
browser. A science challenge problem presents a phenomenon to be explained (e.g., determining the cause of 
the dinosaur extinctions, or of a mysterious disease on Guam known as Guam PD), along with indices to 
relevant resources. For example, one can obtain lists of articles posing possible explanations of the 
phenomenon, reporting empirical findings from fieldwork or laboratory work, or explaining basic domain 
concepts. These are relatively ill-structured problems: at any given point many possible knowledge units 
may reasonably be considered. The materials we used were modified from the classroom versions of science 
challenge problems developed by Arlene Weiner and Eva Toth. 1 The experimental version excluded hands- 
on activities, links to external sites and activity guide. 

P*' ' 



Available at http://lilt.ics.hawaii.edu/belvedere/materials/index.html. 



Computer Setup . The computer screen was divided in half 
as shown in Figure 4. The left-hand side contained the 
representational tool — any one of Text, Graph (shown), or 
Matrix. The right hand side contained a web browser open 
to the entry page for the science challenge materials. 

3.2.3 Procedure 

Participants were seated in front of a single monitor and 
keyboard. After an introduction to the study and signing of 
permission forms, participants were shown the software and 
allowed to practice the basic manipulations such as creating 
and linking nodes or filling in matrix cells. This training did 
not involve any mention of concepts of evidence or of the 
problem domain. 

Participants were then presented with the problem statement in the web browser on the right. The problem 
solving session was initiated when they were instructed to identify hypotheses that provide candidate 
explanations of the phenomenon posed, and to evaluate these hypotheses on the basis of laboratory studies 
and field reports obtained through the hypertext interface. They were instructed to use the representational 
tool during the problem solving session to record the information they find and explore how it bears on the 
problem. Participants were responsible for deciding how to share or divide use of the keyboard and mouse. 
The procedure described in this paragraph was repeated, first with a "warm-up” problem, and then with the 
problem for which data is reported below (Guam PD). Sessions were videotaped with the camera pointed at 
the screen over the shoulder of one of the participants. 

3.3 Results 

Analysis was based primarily on coding of transcripts of participants' spoken discourse, and secondarily on 
participants' representational artifacts. 

3.3.1 Coding and Analysis of Discourse 

Pilot study videotapes from the six one-hour problem-solving sessions were transcribed and segmented. A 
segment was defined to be a modification to the external representation or a single speaker's turn in the 
dialogue, except that turns that expressed multiple propositions were broken into multiple segments. 
Segments were coded using the QSR Nud*ist software package. 

The following codes provide the dependent variables of interest. Epistemological Classification codes 
discourse about the epistemological status of a statement, including classification as empirical (e.g., "that's 
data"), theoretical (e.g., "that's a hypothesis, isn't it?") or discussion of the choice (e.g., "do you want me to 
go data or hypothesis?"). In the present study we only wanted to see whether the tools differed in their 
prompting for making this choice, so did not discriminate these subcategories. Sub-dimension Evidential 
Relation is applied to segments where participants discuss or identify the nature of the evidential relationship 
between two statements. The codes are Consistency (e.g., "it's also for," "that confirms"), Inconsistency ("so 
that’s against," "with this one, no, conflicts, right?"), or Equivocal , applied when participants raise the 
question of which relationship holds, if any, without identifying one specifically ("is that for or against?," "it 
can neither confirm nor deny"). In some cases, evidential relationships were apparently being ^pressed in 
terms of the representational primitives provided by the software (e.g., "connect these two"). These 
utterances were also coded with the appropriate Evidential Relation category, but marked with the Level 
code (discussed below) so that such "tool-level talk" could be distinguished during the analysis. Topic sub- 
dimension Other Topic codes segments not coded as one of the above topics. The "other" codes include On- 
task (e.g., "are we done with this?"), Off-task (e.g., "what's for lunch?"), or Unci ass ifiable (e.g., "uh," 
mumbles, etc.). 

The remaining coding dimensions are used to select out relevant segments for particular analyses. Mode 
indicates whether the segment is coded for its Verbal content or for an action taken on the Representational 
artifact. The final two dimensions only apply to verbal segments. Level is applied only to Epistemological 
and Evidential Verbal segments, and indicates whether an utterance made direct use of epistemological or 
evidential concepts (e.g., "supports," "hypothesis": Conceptual) or was expressed in terms of the software 





i & 


1 < 1 H 


:§§! 
Each of the four 
representational tools f 
would occupy this | 
space in their 
i respective conditions. | 


Mass Extinctions : 

j Problem Statement * 
Background 
ij Ideas Scientists 
| have Had 
| Field Reports 

l 




Ssftk.it ’ <'■. .is * -.V.*, tfSSSa iiifc t ; i : 


Figure 4. Screen Layout for Studies 



(e.g,, "link to this," "round box": Tool-based). Ownership indicates whether the participant was merely 
reading text that we provided (Recited) or expressing their own ideas (Non-Recited). 

Coding was performed by two of my assistants (Chris Hundhausen and Laura Girardeau). Questions of 
interpretation, problematic segments, etc. were discussed among the three of us during meetings, but the 
coding itself was done independently. Inter-rater reliability was computed using the Kappa statistic across all 
of the categories described above, producing a value of 0.92 (n=1942). 



Table 1. Summary of Verbal Coding 



Verbal segments tested: nesting indicates 
subset selection; % are of "Not Off Task" 


Text 


Graph 


Matrix 


N 


% 


N 


% 


N 


% 


Non-Recited 


778 


n/a 


626 


n/a 


537 


n/a 


..Not Off-Task 


694 


100 


613 


100 


508 


100 


....Evidential Relation 


4 


0.58 


32 


5.22 


100 


19.69 


Consistency 


3 


0.43 


21 


3.43 


54 


10.63 


Inconsistency 


1 


0.14 


6 


0.98 


35 


6.89 


Equivocal Evidential 


0 


0.00 


5 


0.82 


11 


2.17 


Conceptual 


3 


0.4 


9 


1.47 


43 


8.46 


Tool-Based 


1 


0.1 


23 


3.75 


57 


11.22 


. . ..Epistemological Classification 


39 


5.62 


57 


9.30 


36 


7.09 


Conceptual 


19 


2.74 


33 


5.38 


7 


1.38 




20 


2.88 


24 


3.92 


29 


5.71 



Selected results of coding are shown in Table 1, focusing on segments coded as Mode=Verbal, and showing 
both counts and percentages for each of the three treatment groups. Percentages are taken relative to Non- 
Recited on task utterances, shown in the second row. Counts and percentages for Evidential Relation are 
broken down in two orthogonal ways: by whether the relation was Consistency, Inconsistency, or Equivocal; 
and by whether the talk about evidence was Conceptual or Tool-Based. Epistemological Classification was 
broken down by Conceptual or Tool-Based. Due to the small sample size we did not perform statistical 
testing in this preliminary study. 

3,3,2 Qualitative observations 

The document created by one Text group contained no expression of evidential relations, and the transcript 
of verbal discourse for this group contained no overt discussion of evidential relations. All of the discussion 
of evidence in Text occurred in the other group at the end of the session (the longest session in the pilot 
study), at which time they also added several expressions of evidential relations. A document produced by 
one of the Graph groups is notably linear, in spite of the fact that Graph is normally considered a nonlinear 
medium. A pattern of identify information, categorize information, add it to the diagram, link it in is typical 
of interactions in this transcript. This pattern of activity, which leads to the linearity of the graph, is 
consistent with the competitor to the Elaboration hypothesis: participants may feel that the primary task is to 
connect each new statement to something else, after which it can be ignored. Finally, the Matrix artifacts 
were especially striking because participants were not specifically instructed to fill in all the cells, yet they 
did so. The transcripts illustrated participants’ systematic identification of evidential relations as they worked 
down the columns, and in one case their appropriate use of the table to rule out a hypothesis ifitft they had 
proposed. 

3.4 Discussion 

Recall that the Search hypothesis predicts that participants will be more likely to seek evidential relations 
when using representations that prompt for these relations with empty structure (Text < Graph < Matrix). 
The row labeled "Evidential Relation" is relevant to the Search hypothesis. This row counts, for each 
treatment group, the percentage of verbal segments that were coded with any one of the three evidential 
values (Consistent, Inconsistent, Choice). The results appear to be consistent with the Search hypothesis: 
Text=0.58% < Graph=5.22% < Matrix=19.69%. This trend holds even when limited to Conceptual 
expressions of evidential relations: Text=0.43% < Graph=1.47% < Matrix=8.48%. Note however that a 
substantial portion of talk about evidence in the Graph and Matrix conditions is tool based (about two-thirds 
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of Graph and half of Matrix evidential utterances are tool-based). This is as expected, since these tools, 
unlike Text, provide objects that may be referred to as proxies for evidential relations. 

The breakdown of Evidential talk according to the type of relation shows the influence of the exhaustive 
prompting of Matrix. In Text and Graph, participants focused primarily on Consistency relations, a possible 
manifestation of the confirmation bias. Treatment was more balanced in Matrix, with almost half of the talk 
about evidential relations being concerned with inconsistency or equivocal relations. This may be because 
Matrix prompts for consideration of relationships between all pairs of items: participants are more likely to 
encounter inconsistency or indeterminate relations when considering those they may have neglected in the 
Graph or Text conditions. 

Addressing the Concept Use hypothesis, we found that 5.62% of Text, 7.09% of Matrix and 9.30% of Graph 
utterances were concerned with the classification of new information as data versus hypothesis or their 
equivalents. We believe that Text would have been lower, except that the instructions for all three conditions 
directed participants to consider and record hypotheses and empirical evidence. Text participants, like others, 
complied with these instructions, for example, by labeling propositions as “Data” or Hypothesis.” Graph’s 
greater proportion of epistemological classification talk is explained by its most explicit use of visually 
distinct shapes to represent data and hypotheses. 



4 Conclusions 

Overall, the results are encouraging with respect to the question of whether there is a phenomenon worth 
investigating. Differences in the predicted directions were seen in both talk about evidence and about the 
epistemological status of statements. However, this sample data cannot be taken as conclusive. Caveats, all 
of which are being addressed by ongoing work, include the small sample size (hence no test of significance), 
the lack of a learning outcomes measure, and the need for a more direct test of the claim that 
representational state affects subsequent discourse processes. Furthermore, analyses based on frequencies of 
utterances across the session as a whole fail to distinguish utterances seeking evidential relations from those 
elaborating on previous ones (i.e., between the Search and Elaborate hypotheses), or to show a causal 
relationship between the state of the representation and the subsequent discourse. A more sophisticated 
coding is required to test whether the representation or salient absence of a particular (kind of) knowledge 
unit influences search for or elaboration on that unit. All of these deficiencies are being addressed in a study 
underway at this writing. Pending the results of this study, plans for future work include attempts to replicate 
selected results in distance learning situations, both synchronous and asynchronous. This line of work 
promises to inform the design of future software learning environments and to provide a better theoretical 
understanding of the role of representational guidance in guiding learning processes. 
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This paper concerns the use of dialogues in student learning and how such 
dialogues can be captured for subsequent use by other learners. The process of 
learning by observing another person* s learning is known as vicarious learning. 
The paper begins by discussing the movement towards more flexible types of 
learning and the belief by many that traditional dialogue has been omitted from a 
lot of today’ s courseware. Dialogue can be considered as one of the stages in the 
learning cycle and to support it there is a need to create tertiary courseware, this 
being the third stage in the cycle. Some of the research that has taken place into 
vicarious learning is described and this has shown that it has some benefit to 
learning and also produces positive feelings in students of being part of a 
learning community. Finally the vicarious learning resources that have been 
produced within a software development course at Edith Cowan university using 
a dynamic screen capturing tool are discussed together with a possible 
dissemination system. 
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1 Introduction 

Universities and colleges today have record numbers of students and yet the cost being spent per student is 
steadily decreasing as budgets are cut and universities become ever more competitive. One of the 
consequences of this is that many managers are turning to the Internet as a means for delivering courseware 
to students in a supposedly cost-effective manner. Students are also demanding more flexible learning with 
learners being able to feam when they want (frequency, timing, duration), how they want (modes of 
learning), and what they want (that is learners can define what constitutes learning to them) [14]. 

The situation has therefore arisen that students spend more time away from a traditional campus and 
technology is being used to provide the necessary flexibility with computer networking empowering 
connectivity and communication, allowing synchronous and asynchronous one-to-one and one-to-many 
communication [13]. However, such technology does not necessarily support some of the learning situations 
that are necessary in higher education. Laurillard [6] points out that learning in many educational contexts, 
particularly in higher education, requires learning about descriptions of the world, knowledge derived from 
someone else's experience, and from understanding someone else’s arguments. She states that: * 

We cannot claim to have sorted out once and for all what students need to be told if they 
are to make sense of topic X. No matter how much detailed research is done on the way 
the topic is conceptualised, the solution will not be found in new ways of putting it across. 

The new way of telling may sort out one difficulty, but it may well create others. All we 
can definitely claim is that there are different ways of conceptualising the topics we want 
to teach . So all we can definitely conclude is that teachers and students need to be aware 
of those differences and must have the means to resolve them. 

The main way this has been done in the past has been by students participating in dialogue with fellow 
students and their tutors. We do have email and synchronous "chat" available to support dialogue to some 
extent but it may well be argued that this is insufficient to support the above. 
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2 The Learning Cycle 

Dialogue can be considered as a crucial part of the learning cycle [9]. The cycle is shown in figure 1 . 
Primary Courseware Secondary Courseware Tertiary Courseware 




Figure 1: The learning Cycle 



It can be considered to comprise: 

• conceptualisation which comes from interacting with the primary content and elates to a learner's 
current state of understanding. 

• construction and the use of knowledge occurs with the use of secondary courseware tools such as 
concept mappers. It involves picking out particularly relevant material, putting the information together 
in ways which have meaning for the learner, and relating old and new material into a coherent whole. 

• dialogue which involves the testing of understanding and can possibly be facilitated with tertiary 
courseware. 

Mayes et al [9] suggest that the third section of the learning cycle, dialogue, can itself be broken up into 
three stages, these being discussion, reflection and reification. Mayes et al agree with Laurillard that 
discussion is fundamental to effective education and that a deep understanding is promoted far more 
effectively and efficiently during discussions. Reflection has always been thought to be an important aspect 
of learning and can be considered as the testing of new knowledge against the schemata that hold our 
existing knowledge. And finally reification is a term put forward by Mayes et al and concerns the structuring 
of newly acquired knowledge into a new object of thought integrated with other knowledge. 

The question then arises as to what sort of tertiary courseware can be produced and utilised to support the 
dialogue aspect of the learning cycle bearing in mind that the material will have to be used in flexible 
learning environments. One particularly interesting line of research has been into recording of discussions 
and making them available to other students in a flexible mode. This concept is known as vicarious learning 
where this is defined as [2]: 

, The potential benefit to learners of being able to observe or * listen in ' on experts or their peers as 
they discuss a new topic . 

The following can be considered to be vicarious resources: 

• Frequently asked questions (FAQs). Here students can learn from the answers to typical questions posed 
by other students. 

• Listservers. These promote vicarious learning as students receive the text dialogues that take place 

between various subscribers. The term “lurker” is often used for the person who does not participate in 
dialogues but prefers to simply observe. ^ 

• Bulletin boards. These provide the means for asynchronous dialogues and again can be used by 
“lurkers”. 

• Chat rooms. These provide the means for synchronous dialogues. 



3 Research into Vicarious Learning 

Research initiatives are in two main areas, the first attempting to determine if vicarious learning is of benefit 
to students and the second looking at how such dialogues might be made available as tertiary courseware for 
re-use by other students. 



There are several interesting questions that might be worthy of investigation in the first area. Cox et al [2] 
suggest that we need to determine who are useful models for the vicarious learner, experts or novices. It 
might be better to observe experts as skilled behaviour would hopefully be modelled in a clear way, although 
this is not of course always true as many experts find it difficult to make their knowledge explicit. It could 
be argued that student - student dialogues would be better to observe as the observing student would be 
better able to identify with other students. Also the students participating in the dialogue might use more 
appropriate language and also ask questions of each other that they may not have wished to ask their tutor. 
Cox et al also point out that observing unskilled behaviour may also prove to be of benefit as the observing 
student would determine from the dialogue what sort of errors to avoid without having to make those errors 
themselves. Also of course, the dialogue type to observe may depend on the type of student who is the 
observer. It might be more appropriate for a strong student to observe experts and for a weak student to 
observe novices. 

In one particular piece of research on vicarious learning [7] benefits were found that were both cognitive, 
with an increase in knowledge and understanding in the particular curriculum area, and social with exposure 
to peer discussion creating positive feelings of being part of a learning community. 

Lee et al [7] carried out research within an on-line Masters level course in Computers in Teaching and 
Learning. They created task-directed discussions (TDDs) in order to capture good learning dialogues 
amongst students and to overcome the “barriers of silence” that might otherwise occur. Over 30 hours of 
discussions among students, and between students and a tutor (the expert), using the TDDs were videoed. 

An architecture called the Dissemination System (DS) was created from primary instructional materials and 
integrated clips taken from the videos. The DS allows a multimedia database of video and audio clips, text 
transcriptions, and annotated graphics to be integrated with primary expository teaching material and 
delivered via the Web. The system was then used in an experiment to investigate the vicarious resources in a 
controlled laboratory setting. 

The experiment used a section of the course on Models of Learning with Technology. Two sets of learning 
materials were created, the first comprising primary learning materials (approximately 45 web pages) and 
the second comprising both primary learning materials and an integrated set of vicarious learning resources. 
The vicarious resources had been obtained from the videoed dialogues and comprised 108 video clips, 13 
audio clips, 43 text transcriptions, and 27 audio annotated graphics. The resources were accessible by either 
clicking on highlighted keywords or by a search mechanism. 

Two groups of students took part in the experiment, one using only the first set of learning materials whilst 
the other used the second set of learning materials which included the vicarious resources. The conclusions 
that Lee et al drew from the experiment were that there were some benefits in learning and substantial 
positive changes in attitudes and discussion behaviour for the students who used the vicarious learning 
resources. The researchers also make the point that although some people claim that learning can only take 
place when students are personally engaged in discussion, the evidence suggests that observing peer 
dialogues can, on the contrary, provide a useful source for learning, both cognitively and socially. The 
researchers have in fact suggested that such vicarious learning may sometimes be more beneficial than being 
a participant, depending on the state of the learner [11]. 

The web based materials used in the experiment are available at http://www.hcrc , ed.ac.uk/Vicar/TT/ . They 
are fairly slow to download from the Web but realistically they could be put onto a CD ROM for use with 
distance learners. The audio dialogues that are available are played whilst a static graphical image is 
displayed to the learner. Such a dialogue concerns the graphic being displayed and I felt that something was 
lost in this type of dialogue and that it would have proved to be more useful and meaningful if objects on 
the graphic could have been “pointed to” in order to draw the observer’ s attention to the important aspects of 
the graphic. 



4 Creation of Vicarious learning Resources with Dynamic Screen 
Capturing Tools 



During the summer school of 1998 at Edith Cowan University, I made use of Lotus ScreenCam for student- 
tutor dialogues within a Software Development unit. Between lectures and laboratory sessions, students had 



no contact with me as I was off campus, however I did have access to email at home enabling students to 
send me ScreenCam movies of any programming problems that they were having. In addition to movies, 
students would also send the programming code enabling me to use this when making a “reply” movie. An 
example of a screenshot taken from a movie, which was sent to me by a student, is shown in figure 2. 
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Figure 2 



The movie had several text captions and concerned a problem that this student was having with passing 
arrays to subprograms in Visual BASIC. A screenshot taken from the movie, which I made and subsequently 
sent back to the student, is shown in figure 3. 
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it at the beginning of the 
procedure .W*;'' 




le (Dir$ (FileName) ■ 
igBox "Can't Find the File! ...Enter Again", , "In£c«J( 
it FileNarae = InputBox ( "Enter Full Path and File Mr n * 
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— ha 



leNowe = LCase (FileName) 
unt Value = 0 



Dim Stocklnf ci ( 1 To 100) Ah StockType 



Let Flag ■ "Y" x 

Do Uhile ( (FlleName) = ( "a: stock. txt") And (Flag e Zi")} 
HsgBox "File Found!!! Enter Stock Information f/ow ", 



Do Uhile (Flag “ "Y") 

Let CountValue = CountValue + 1 

Let Scocklnfo (CountValue) . Stock ■ InputBox ( M EnterH 



Figure 3 



The screenshot in figure 3 includes a text caption that has nothing to do with the original student problem. It 
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is the sort of comment that I would make if I were looking at the code that a student had produced in a 
laboratory session. In the rest of the movie, I was able to make suggestions on how to overcome the original 
problem and I also included a captioned comment about the lack of comments within the student’ s 
programming code. By using ScreenCam, I had been able to engage in a richer asynchronous dialogue with 
the student than I would otherwise have done by conventional means. In addition, as a side effect, I was 
building up vicarious learning resources for use in future semesters. 

In addition to capturing asynchronous dialogues as described above, Lotus ScreenCam can be used to 
provide rich feedback to students on their assignment work. Simple “low-tech” audio tapes have been used 
in student feedback [1] and it is suggested that such feedback adds a social dimension to the commentaries 
with the tutor being able to talk personally to each student, whereas written comments lacked context and 
sounded impersonal. 

I produced a set of such movies for the small group of campus-based students that were involved in the 1 998 
summer school session mentioned earlier. Each week the students attempted a small programming problem 
and handed in the relevant programming code together with a small text -captioned movie explaining their 
program. I then made a feedback movie for each student. Each feedback movie had audio commentaries to 
keep the production time to a minimum and the movies were placed onto ZIP disks that had been provided 
by the students. I was able to go through the programming code on the screen, highlighting areas of interest 
with the cursor whilst making comments and in addition run the student programs with a variety of data 
whilst passing comments about both the good and the bad points of the programs. 

The sets of movies that the students handed in and that I produced have now become another vicarious 
learning resource for use by students in subsequent semesters. Each week, students are given a small 
programming problem to attempt and they can then use the movies to view the student - tutor interactions 
for a similar programming problem. In practice, students have commented on how useful they have found 
these resources. Feedback was elicited on-line and some of the comments follow: 

• I found it helpful and interesting in giving clear visual instructions or explanations. 

• All the other students solutions were very helpful. And they were informative. 

• Only used the movies once, but they do provide a good resource for students experiencing difficulty. 

• Pick up other students mistakes. 

• Always forgot how to get to them 

• Probably slack, but using the sound was too much hassle. 



5 Delivery Mechanisms for Vicarious Resources 

The last two student comments above indicate that there is a need for some form of technological delivery 
mechanism for the vicarious resources that have been produced that is simple and easy to use. Students need 
to be able to quickly find movies that are appropriate for the programming problem that they are attempting 
and then view the movie. We have experimented using the Web to deliver the movies however this has been 
a problem as movies with audio are of the order of 1MB in length per minute and take too long to download. 
Realistically it is necessary to make the movies available on CD ROM and we will be using a Windows Help 
file as a way of delivering the movies. There aie several Help file authoring tool available amfone that I 
have used extensively is ForeHelp [4]. A Help file can be produced with the usual contents andmdex pages 
with little effort and programs can be launched seamlessley thereby permitting the running of ScreenCam 
movies. 



6 Discussion 

It would appear that the use of vicarious learning resources by students can benefit learning and also provide 
positive feelings of being part of a learning community. However the creation of such resources needs to be 
done very carefully so that they are relevant and of interest to learners. If a synchronous dialogue is to be 
recorded by the use of video or audio then it is important to use task directed discussions [7] to ensure that a 
relevant dialogue ensues. Asynchronous dialogues usually take place by email or bulletin boards, however 
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they can be made richer if a dynamic screen capturing tool is used. Finally the vicarious learning resources 
that have been collected need to be made available to other learners and to this end Lee et al created a web 
based dissemination system. Another approach is to use a Windows Help file for disseminating such 
resources assuming that delivery is to be by Wintel hardware only. 

In the future I intend to look at capturing synchronous dialogues using a dynamic screen capturing tool. 
These would be both student - student and student - tutor where the two participants sit in front of a PC 
whilst having a dialogue concerning a program that is being displayed. 
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This paper proposes a method to localize a feature extraction area for 
touch-type training using a camera. The feature extracted from pictures of a 
trainee’s face is used for recognition of one’s eye direction. The recognition of 
a trainee’s eye direction enables us to give a trainee warnings and appropriate 
advice, and these warnings and advice assist a trainee to learn touch-typing 
quickly. The goal of this study is constructing a system which supports 
touch-type training using a camera. In this paper, a method to localize a feature 
extraction area using a horizontal direction’s histogram and a vertical one 
mutually is proposed. These histograms are made from the sum of darkness 
values of pixels. Experimental results of the proposed method with trainees’ 
black-and-white still pictures are described. 

Keywords: Touch-type training, Camera, Feature extraction, Histogram, 
Pattern recognition, Eye directions 



1 Introduction 

In this paper, we propose a method which localizes a feature extraction area to recognize a trainee’s eye 
direction for touch-type training. By the recognition of a trainee’s eye direction using camera, it is possible 
to warn and give advice when a trainee looks at improper places. These warnings and advice must be 
effective for trainees to learn touch-typing quickly. 

The capability of typing quickly is very helpful in studying and using IT (information technology). 
Nowadays the capability is becoming more and more important because many people use computers for 
calculaling, writing, reading and so on, and the keyboard is the most common device to input characters. 
Users can type on a keyboard and train themselves in their own manner, because the keyboard is easy to 
use. However, typing in one’s own manner has a speed limit which is much lower than typing in a proper 
manner. Furthermore, users’ own training ways are not adaptable, that is, using their own typing style, the 
users would be able to type certain words or sentences quickly, but not so many. 

Some training methods are proposed[l]-[3] for the purpose of learning touch-typing. Although the details 
of the methods vary, the methods have a common point. The point is that after a trainees roughly learn key 
positions, they should look at a display and should not look at a keyboard in order to learn t^ych-typing 
quickly. 

The goal of this study is constructing a system which supports these touch-type training methods using a 
camera. The system warns and gives appropriate advice to a trainee automatically when one looks at 
improper places. In this paper, we deal with classification of eye directions into the following three classes: 
Looking at the display is the first class, looking at the keyboard is the second, looking at other places (not 
the display or the keyboard) is the third. Especially, in this paper, we propose a method to localize a feature 
extraction area for the classification using histograms of a trainee’s face pictures. These histograms are 
made from the sum of darkness values of pixels. 
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2 Touch-type training system using a camera 

2.1 Structure of the system 

The supposed touch-type training system is as follows. 

Hardware: A personal computer with a CCD camera. 

Software: A touch-type training program and an eye direction recognition program 

2.2 Process of an eye direction recognition program 

Extracting an eye area from a picture of a trainee’s head 

Taking a picture of a trainee’s head or the upper half of one’s body using a camera which is placed on top 
of a display in the middle, and extracting an eye area from the picture. 

Extracting features and classifying an eye direction 

Extracting features from the picture of a trainee’s eye area, then classifying the eye’s direction into three 
classes. 

Warning and advising 

Warning with sound and giving appropriate advice when a trainee looks at improper places. Warnings 
concentrate one’s attention on the training, and giving advice is effective for fast learning. 

Some other research proposed methods on the process (1) and produced good results [4]. In this paper, we 
deal with the process (2) on the supposition that the process (1) is already done. To be applicable for most 
of touch-type training methods, the process (2) classifies eye directions into the following three classes: 
Looking at the display, the keyboard, other places (not the display or the keyboard). 



3 Feature extraction and classification 

Figure 1 shows vertical and horizontal histograms used for the classification of eye directions. The 
histograms are made from the sum of darkness values of pixels. Some research used the histograms, and 
reported that they are effective for the recognition of eye direction [5]. 

A vertical direction’s histogram in an eye area has basically two peaks: the upper peak is the eyebrows; the 
lower one is the eyes. We expect those two peaks are key features for the recognition of eye direction, 
whether it be up or down. 

On the other hand, a horizontal direction’s histogram also has two peaks. Those peaks are expected to be 
key indicators for the recognition of eye direction, right and left. 

In this study, a template matching method is used for the classification of eye directions using vertical and 
horizontal direction’s histograms. 

4 Localization of a feature extraction area 

To obtain high classification rates, we must extract features from the area available for the classification, 
or reduce noise which is unavailable for the classification from a feature extraction area. For these 
purposes, we propose a method to localize an eye area. 

4.1 Localization in a horizontal direction 

Most vertical histograms have two peaks. The lower peak is basically at a line of the two eyes as shown in 
figure 2(a). Thus we localize an extraction area at the lower peak, and get a horizontal direction’s 
histogram from the area as shown in figure 2(b). The histogram is expected to be a key feature for the 
recognition of right-and-left eye direction. 
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A vertical 
direction 




Figure 1. Histograms in an eye area. 




(b) A localized horizontal 
direction’s histogram 



Figure 2. Localization in a horizontal direction. 

4.2 Localization in a vertical direction 

We localize a feature extraction area in a vertical direction using the localized horizontal direction’s 
histogram (shown in figure 2(b)). 

Peaks of the localized horizontal direction’s histogram (shown in figure 3(a)) are basically at the two eyes 
(figure 2(b) is equal to figure 3(a)). Hence, we get vertical direction’s histograms as shown in figure 3(b), 
using tbe horizontal direction’s histogram as shown in figure 3(a). These histograms are expected to be key 
features for the recognition of up-and-down eye direction. 

4.3 Experimental results 

We made experiments to evaluate the proposed method of localization as follows: 

- First, taking twenty- four still pictures of the trainees’ heads (Three pictures were taken of each of eight 

trainees from each class(described in sect.l). The pictures were black-and-white, 256 picture elements 
and 72dpi.) 

- Then, extracting an eye area by hand. 

- Finally, localizing feature extraction areas as mentioned above. 

Table 1 shows experimental results of localized areas by the proposed method. In table 1, the center of 
eye(s) means an iris in case an eyeball was shown in the picture, or the center of an eye’s outline in case an 
eyeball was not shown in the picture. 
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(a)A localized horizontal 
direction’s histogram 
(equal to fig.2(b)) 



(b)A localized vertical 
direction’s histogram 



Figure 3. Localization in a vertical direction. 



A localized area 


A horizontal direction 


A vertical direction 


the center of two eyes 


2 


10 


the center of an eye 


13 


12 


others 


9 


2 



Table 1 Experimental results of localization 

The reasons why the localized area was not at the center of two eyes are the following: 

- 70% of the total were caused by inclined heads. 

- 30% of the total were caused by noise such as glasses or hair. 



5 Conclusions 

In this paper, we propose a method to localize a feature extraction area using histograms for touch-type 
training. The feature is used for recognition of a trainee’s eye direction. 

Experimental results show the method has some problems when the following conditions exist: a head is 
inclined, noise such as hair or glasses are shown in the localized area. 

To solve these problems, we have the following plans: 

- Localization is carried out after adjusting inclined heads. 

- Selecting one eye in case the localized area is at the center of only one eye. 
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An informatization society, where high added value can be created through 
networks is different from an Industrial society where physical labor predisposes 
discrimination between men and women. As knowledge and information are 
regarded as the most important resources in an Informatization society, 
intelligence and emotional ability are given more importance than physical 
superiority. In light of this, the roles and positions of women are being newly 
evaluated, and the direction of the women’s informatization movement has 
emerged as a new topic of the era. Korea places a strong emphasis on education 
and the percentage of women who attend University is also high. Contrary to 
this, the percentage of highly educated women who become working members 
of society is very low. These days, this problem has been recognized and 
education in woman informatization has actively proceeded. Taking various 
kinds of women's organization as the principal axis, woman informatization 
projects are actively unfolding. Government has also explored supporting plans 
in various aspects. This study explores the present state of women’s 
informatization education and it's future direction. 

Keywords: Woman Education, Informatization Education 



1 Introduction 

An Informatization Society is expected to be the 3F era: Female, Feeling, and Fiction. Productivity of 
culture and emotions is more important than labor productivity. Instead of labor productivity, sensitivity and 
intuition unique to women are expected to contribute a lot to the development of an emotional business that 
has high added value. 

As the roles and position of women are newly evaluated, the direction of the women’s informatization 
movement is emerging as an important topic. As it becomes necessary to have women’s informatization 
education in Korea, a new movement has emerged. In light of this, this study will explore the present state 
and future direction of informatization education of women in Korea. 
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2 Present State and Problems of Woman Informatization Education ini 
Korea 

The studies on women and the informatization society have been introduced in Korea since the end of 1980. 
Until now, works on the development of women in an informatization society have been produced 
intermittently. The Korean Women’s Association and National Women’s Convention has dealt with “Woman 
and Informatization Society," and attracted the attention of the society of women overall. However, 
systematic, continuous and comprehensive studies on women and informatization have been insufficient. 
Also insufficient are the studies on the concrete policy alternatives for informatizing all women in 
accordance with national informatization. 
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